Announcer: It's Tuesday, June 26th 2007, and you're listening to ARCast.
Ron Jacobs: Hey, welcome back to ARCast. This is your host, Ron Jacobs, and today we are going back to Wales where I was at the UK Architecture Insight Conference. I had a really interesting chat with a couple of guys, Keith Walker and Peter Goss, who are working on a project for the London Underground. A lot of times people have said to me, "WPF looks kind of cool and snazzy, but where would you use this powerful, 3d visualization technology? What would you do with it?"
I'm sure there's a lot of ways people will use this, but here's a great example: You just imagine in the control room of the London Underground. You're looking at maps of the system and you want to be able to visualize what's happening in the network. Well, that's what these guys are doing. It's a pretty fascinating project. I think you're going to enjoy this, so sit back and let's welcome Peter Goss and Keith Walker.
[applause]
Ron: This is Ron Jacobs back at the Architecture Insight Conference here in Wales. We're having a lovely time, just sitting around for a chat, as they say here in the UK. We don't say "chat" very often back in the US.
Keith Walker: You don't?
Ron: No, we don't.
Peter Goss: What do you guys do over coffee?
Ron: Oh, yeah. We talk.
Peter: They simply negotiate.
Ron: We talk. When I tell people that ARCast is like a talk radio show, they, "Oh, it's like a chat radio." I thought that was kind of interesting.
Anyway, that's beside the point because today we're going to talk about some of the interesting work you guys are doing with regards to visualization of the Tube network in London. I have to say, of all the things I like about London, I think the Tube is pretty cool.
Peter: OK.
Keith: Well, that's a first!
Ron: Maybe London people don't like it, but I think it's fantastic because every time I come to London I can get all over town. I totally like the environment and being able to get around, and Seattle doesn't have anything like that. If you come to Seattle and you want to get around, you have to take taxis or cars or whatever. We don't have anything like that. Maybe that's why I think it's so cool.
But tell me about this project: what did you guys do?
Peter: OK, to start from the beginning, I guess--
Ron: Oh, wait, wait, wait. You know what? We didn't even introduce you guys.
[laughter]
Peter: Well, let's start at the very beginning, then.
Ron: Let's start at the beginning. This is Peter.
Peter: I'm Peter Goss. I work for Solutions Development and we create the operational software for the London Underground. We work in a team called Common Services and Keith works with me.
Keith: I'm from the main solution design; I'm the techie guy within the department. We serve the entire department, and help to make sure they can write the systems they need to and classify designs. We also look at R&D type technologies, and that's partly why we've engaged Microsoft to do this bit of concept work. It's actually to look at WPA from.Net three and see how we could then use that technology to help us realize some challenges that we had in terms of technology.
Ron: OK, so being that this is ARCast, we should first talk about the challenge or the business need that you were out to try to solve. So what was that?
Keith: Basically, we've got a number of systems already which shows in managing in the network and shows the occupational railway. We've a good number of systems which show the service quality and shows us how often the trains are coming through the stations. We've got a system called Trackernet, which is using the technology of SVG, which actually then shows you line by line, where the trains are, the points, the signals, the crew on those trains.
We've also got some other systems as well which show you a two-dimensional map of a network and we can overlay incident dates of things like engineering works, closures, SPADS, which are signals passed at danger, crime incidents, and that way you can actually see visually what's the response.
These are all separate, disparate applications. One of the main problems with tracking that we have, is that it only shows you a single line at a time. So what we're going to do is write an application that can actually show you the entire network, where the trains are, across the whole network, and allow them to overlay more information that what we have currently with TrackerNet.
A bit of a limitation using SVG, there were certain things like pixel width. So there was only so much that we could get onto a single canvas, if you like. We're then looking for a new GUI strategy, so we had about four or five different options that we had to narrow down, evaluate, and one of those options was the WPF.
Peter: There were a number of things that got us to where we are today. I guess there were a lot of business requirements coming back from about two or three years ago when we tried to put a development and architectural strategy together. It all began with, I guess which is a common aim for a lot of companies, was to have a single entrance to all of our applications, and so we went along the SharePoint Portal 2003 route.
Great idea, we got all of our applications together, but there's not really much cross-functionality. Each individual application has an individual function. Yet lots of them can overlap and there are lots of linked data there which the operating staff could use if they had the opportunity.
Ron: Yeah.
Peter: Our applications, all ASP.NET, a lot of hard work would be required in order to do that in that in the current technology. So we're looking for this new GUI strategy and trying to develop a platform that we can create all of our applications going forward with. So we've got the opportunity to work with Microsoft and do a three-week Microsoft Technology Center proof of concept, and that's where it all began.
Ron: So a lot of people may not know how the Microsoft Technology Center works. You guys actually went out there, worked with some people from Microsoft for three weeks?
Keith: Yep.
Peter: Yep.
Keith: Yeah, basically they gave us a lead architect, of course architect, and we supplied three people. There were also designers there from Microsoft as well. There were actually two people, but the equivalent of maybe one full-time over three weeks. So the designers -- well, the user experience people -- went away and worked in parallel while we were doing development work. Towards the end of the project their design work and our development work came together and then the application that we've got now, I think is absolutely amazing. It's really quite astounding.
Ron: That's a really interesting point there, that you said the design work and the development work were going on in parallel and coming together, because that's one of the key goals of WPF. Was it working for you?
Keith: Absolutely, that was one of our main proof points, if you like, of what we're trying to achieve. As a main goal was that we could actually then have a separate design and development work and then be able to bring it together and it really did work.
Towards maybe two days from the end, I was starting to panic a little bit because what we'd done was a typical developer application that had no real gloss on it. It was just very plain. It looked like a developer had done it basically.
Over the last couple of days the two came together, and it was just absolutely stunning. I've never worked on a project with such a response. In the business, outside the business, general public -- because in Britain, everyone knows about the underground, it's something quite tangible that people can be looking at, they know what it is. It's tangible to see this application is quite... It's a very impressive app.
Ron: Wow. This application then, as you described it, takes a lot of data from different background systems...
Keith: Yep.
Peter: Yep.
Ron: ... through web services? Through WCF?
Keith: Yes, it's not WCF yet, but it's all web services. We've spent the last possibly two years, two and a half years, building up our applications so they're multiple tiers. Generally, we expose any data through a web service. So we handle these services there, on top, basically. So probably clients usually write a visualization application, and write another web service to give us some data to basically give you the vector graphic representations, where the things should be placed on the calendars.
But because of those services, they're tracking it to give us chain of movements, instant data to give us crime, sort of engineering works closures. We have also common data which gives us meta data about stations and lines, and the relationships between them.
So really, it was because we had that investment in SOA that we were able to achieve what we did. If we didn't have those services, we couldn't have done what we did.
Peter: That's part of, like I was saying, this two- or three-year strategy. All of our applications, all of our designs have been very loosely coupled. We've tried to create some business services, and there's some other key business services which are coming along. Specifically, our topology service, which one of our portfolio's is putting together at the moment. It's trying to get logical chunks of information which many applications require, and we've been able to take basically all of that hard work over the last two years, and literally slap the GUI and a small application layer on top of that, and get all of that information, and now all of a sudden we can layer different bits of information dependent on who's interested in that information, and it's very, very meaningful.
Because it is very pictorial, it's very rich, and it's something that we couldn't really create in ASP.NET. You couldn't really create controls that give you that same kind of rich functionality.
Keith: It puts a lot of context to data. Service calls is something I mentioned earlier. That's something that's very, very important to, is it's not really a timetable. We don't really care if the trains are running late, or if they're not on time. But what we do care about is how often they're coming. You want them there every two or three minutes. So, for example, what we had before was the topological map. It's the map that most people would know as the underground map. It's not actually shown to scale. The distance between the stations is not the actual distance between the stations.
So what we did was we actually created two different views. A logical view of the map is the passenger map, and then a geographical view. And that view actually allows us then to place trains, and see the physical separation in terms of actual distance. So then the lane managers can look and see there's a large gap there, which then they can then reroute trains to say, "Can you stop, drop the passengers off, then turn around?" And that will ensure the trains are coming more often. Those are things they couldn't do before.
Peter: Then you can enjoy your journey across London.
Keith: Absolutely. That view is really sort of down to the physical tracks. That just shows you the points, the signals, and the actual physical tracks that say things, and everything. So we have these two different views, really, from one set of data. It's been quite straight forward for us to do. It's had this... not extra context, it's the ability to actually see... for example, you can go back in time.
We have archived data of all the train movements. So we can go back two days, for example, and they can run through and see all the train movements. At the same time, you can overlay incidents that occurred. In a number of different systems, you log closures, or incidents. Someone pulled the passenger emergency alarms, or an escalator was closed, and you had to close the station; any number of things. What we've now got is the ability to show you the trains running, and then popping up instances to say, "There's been a problem here," and then they can then look and see how that instance affects the operation of the railway.
So you can actually see an emergency alarm was pulled, and a train stopped somewhere, you can see the trains backing up behind it. And that then is a useful training device for them to work out what would you do in that scenario. It adds a lot of context. What they'd have to do before was have three or four systems open at the same time, and then they would look at the instance that happened at half past six on a Friday night, and they'd have to go back to it fifteen minutes beforehand and play it through.
But because you could only see one line at a time, you could only see how it affected that one particular line. But potentially, if a station's closed, it could also have an impact on other lines that go through that station, because you have intersection stations where you have multiple lines.
So this adds real context to data, whereas before you had to flip back and forth between lots of systems - it was a really time consuming process. It was one of the main things our main directors do in the morning of a call -- it's called the "morning call." They'd look at the instance from the previous day, and how it affected the running of the railway. The time blockout, anything they could stop it from happening, or how, in the future, what decisions they could have made to make it better, or make sure the service is running better.
So it's become a really useful tool, or potentially become a really useful tool. As a proof of concept right now, it's a very useful tool, but we've still not got it into production yet. It's something that we're still working on to get the infrastructure up to speed, and being able to get it into a state where it's production ready. It was a proof of concept, so it wasn't written with error handling, and...
Ron: Yeah. You know, the one thing that I can imagine for something like this is that when I imagine the map of the tube system that I've seen, there's a lot there. That's a lot of data.
Keith: Yes, yes.
Ron: It makes me think that performance can be a real challenge here. If I'm looking at this whole big system, and all the trains, and all the incidents that are happening, wouldn't that be really slow?
Keith: You would think. Potentially. And also, we're basically tracking potentially about 400 trains over 270 stations, over 13 lines. There's a lot of information there, but it's just a matter of tuning, really. The main application works. We have clock service. A timer, basically. Every two seconds it makes a poll to get data, but you can then run it up to 20 times speed, so basically it does it every two seconds.
If you run it at one times speed, then every two seconds it will get two seconds worth of data, then move the trains about two seconds worth of data. But then if you move to 20 times, every two seconds - I'm sorry, four times worth of data. It's just really a matter of tuning. How you want the service to work, and how you want it to render.
We have started to hit some problems when we started to get to 20 times, because you move the data you're bringing back, and you're processing, and you're trying to do this every two seconds, so you're trying to keep the trains moving. But it's actually really surprised me how well it's actually performing.
Peter: It's held up very well, hasn't it?
Keith: Absolutely, yes.
Peter: I mean, you haven't done any really tuning...
Keith: As you can see from proof of concept, it runs I think fantastically well, and we spoke to the guy who came over from Redmond during the proof of concept, and he looked at what we're doing to try and ratify what we're doing, and he has the same questions about performance and what we're thinking about.
He was telling us about other ways that we could potentially look at doing the rendering, rather than the standard out of the box, there's another method of overwriting the rendering aspects of it. It's almost like changing the frame rate of the animation, if you like. So we haven't even looked at that yet, but it's still running well.
Peter: There's an awful lot you can do with it.
Keith: Absolutely.
Peter: There's a whole load of other aspects we can actually incorporate in the application, and three weeks is a very short space of time. And actually, a large chunk of that was actually getting all the data together, and understanding how we can consume that data through that application.
There's a whole number of things that we can potentially do, and link to all the other systems, and also the other modes. We have buses that are going past. We have all the services, we have the taxis, the waterways, we've got the airlines, and all those systems can somehow be linked together. And in a sort of scenario where the train station fails, and what's happened, an escalator's gone down, we could overlay a CCTV camera that's on top of the escalator to see that it's broken. Or maybe there's fire, or there's smoke. We could alert emergency services. We can call the bus company, and say, "We need 20 buses. We have 1,000 customers here that need to go somewhere", and the ability to overlay all this information is potentially very exciting.
But, around here, it's an ambitious proof of concept. We are where we are. We've been able to develop something ever so quickly, and that's very exciting for it, because it allows business to go away, use their imagination, and come back with some solutions. And all of a sudden, we can say, "Yes we can do that for you." And we can do that in a relatively quick timescale and, therefore, at a relatively cheap total cost of ownership of the system.
Keith: I mean a good measure of performance problems I see right now are more in the web services that are being consumed. Because the way that the applications were run previously, web services are being called less frequently, which potentially, if you were going to have a lot of polling, a lot of doing calls, to try and overlay this information, instead of it being one application inside someone's one web service, or maybe two or three, know this is consuming potentially 10, 15, 20 web services at the same time.
This is a lot of network traffic. And potentially, then, the web servers themselves might then obviously need to have some sort of tuning on them. Look at caching on the web services, which weren't there before. So I think we're going to see performance issues rather than within the actual WPF application itself, the system is working.
Ron: Yeah. That's a good point. I did an episode of ARCast with the Commonwealth Bank of Australia guys, and they had built up a number of web services that they use in their smart client application. And they have this composite smart client. And so there's these different pieces, independent pieces, that are calling into a service, and sometimes these services are calling yet other services.
Keith: Yeah.
Peter: Yeah.