I can just talk a little bit about our experience here with the few examples or the Council governments. We've only worked with the Waze data that's real time type data that comes in. The other crowdsource data has been archived. So, the reliability side of that makes it a little easier when it's archive data then when it's live because that's when you really want to respond quickly to that information that's coming in. We do have Waze like I mentioned that comes into the 911 dispatch centers. Then we also use Waze as part of our 511 VFW.

We have not had an issue with reliability of that data source. It seems to come in very reliable from the wayside, so I would say, it's very reliable. I'll also throw out there if you do encounter crowdsource data, that maybe is less reliable, I would think that I would ask the question: But is it still more data that you get from than you have now? Look at your operation of your system. Other crowdsource data outside of Waze there might be holes in some of that data, but I still feel it brings in another source and a lot more data that allows you to look at your system than maybe what you currently have.

So, I would say that they're kind of two different ways of looking at crowdsourcing platforms. One is as a tool to push out information from the agency to the traveling public, and the other is an intake mechanism for gaining and insights from the traveling public, so that agencies can use that for their operational capabilities. So as far as I think, maybe this question is getting at how to get the public to use the crowdsourcing platforms that you have and obviously the bigger the sample size, the more people within your agency boundaries know about it. Then the more able you are to get information and insights from them.

A lot of the crowdsourcing sources that we're talking about are what I would call passive crowdsourcing, and that the person is not really actively having to do anything. With your smartphone, navigation device, or even your vehicle, is collecting information passively without the traveler having to push a button or do anything. So, it's not so much teaching the public how to use that where a lot of these passive crowdsourcing so sources. There are other types of crowdsource data that I would say are not passive, but are active, and that the user or the traveler has to actively do something. This is where the education component can come in where we are asking the public to please help us by using a Twitter handle with a certain hashtag. This can be done through advertised and outreach through your public information office, or something like that. Or there could be a new app we created, and we want people to use it to help us identify where there's weather events or where there's road closures through a 311 system. So again, those kind of mechanisms which are more actively crowdsourced where the traveler has to actually do something and push a button or report something that's really where it becomes kind of an outreach effort within your agency. This gets into distracted driving and making sure that you design the feedback mechanism, that it's not going to cause a safety issue, that they encourage them to do it hands free, or what the passenger or after you stop your trip.

Most of the crowdsourcing data sources we're talking about are passively collected. So the travelers are actually providing that information without having to actively do something, and so that reduces the burden that you have as an agency to have to teach them how to do something.

This reflects on one of the things that we talked about with different agencies. Should we do away with the field equipment that we are using to collect data on our own? This gets into the point of if you have data that you are collecting on your own, and you're getting crowdsourced data in that same area that is providing a similar type of information, you have a perfect opportunity to do some validation. Give yourself some reliance on how well the crowdsource data is. This has always been a topic area when it comes up.

I saw that Mena responded from VDOT. They've done some periodic validation, and I'll put a plugin for the Eastern Transportation coalition as they have done a great job over the years validating various crowdsourcing data sources from probe data. You can go to their website, Eastern Transportation Coalition and see their reports and the methodology that they've used. We have seen a lot of agencies getting into it as a pilot project, and they may hire the local university to do some kind of validation for them. It is a great way to engage the university partners.

From the audience, Stephanie Marik from OhioDOT stated that, yes, we use our INRIX Probe data for two categories of the TOAST calculations. The two are Bottlenecks and Travel Time measures that use the probe-based data. I would point out that Natalie mentioned that Ohio is actually going to be involved in one of our future webinars.

I guess that would get into information about transit reliability on time arrivals and such.

I would use this an opportunity to get on my soapbox, making sure that with crowdsourcing that you identify a need or a problem upfront that you need help with. There is maybe an issue that there's a community with new or expanded transit service. I would try to drill down further on that and find out what exactly is the need or the problem that you would be trying to solve? Based on that, that would help you identify how crowdsourcing data can help you. Is the need to be able to better identify on time arrivals or passenger demand, or the full trip from walking into the transit service and getting off? I would try to better identify what the problem is first and then you can look into how crowdsourcing can help you.

One of the things that comes up the most is with a system like Waze, you can get a number of indications of an incident, and they may show up in a number of different geographical areas. It is when people get around to recording the incident then it gives their location of where they're at when it was reported, but it may be after they've gone down the road a bit. So, that is definitely one of the factors that an agency has to deal with in getting this type of information. I know Kentucky and a couple of other states have looked into how to filter the data that is coming in. There is also the ability to have a score that is associated with the information coming in on the reliability factor of the information coming in.

The only thing that I would mention, is that I agree there are usually multiple indications of when a crash would occur, so you have to be careful to merge those into one crash or maybe it's a primary crash and a secondary crash. They call them Wazers I believe, the users of Waze. They have a score for themselves depending on if the events are verified or not. I think within out applications we look at that rating of those users that we filter out of what would be inaccurate.

Probably the underlying thought is that you'd see those in the younger generation are more apt to be sourcing the data just because of their capabilities with different technology. I haven't really seen that come out as any particular report from the DOT as where they assume they're crowdsourcing data is coming from. Much of that is going to be known from the source of the data as opposed to the DOT itself.

There are statistics out there about smartphone ownership, since a lot of crowdsourcing comes from location-based services with people with smartphones. So, there is public information out there that smartphone ownership overall is increasing by all demographics. There is still a gap by the generation older versus the younger generation so there is potential for sampling bias there. This depends on what you are using the data for. There's also a gap by income group, higher income groups versus lower income groups with smartphone ownership.

There is always the underlying expectation from the transportation agencies that the data that they're using in these situations does not have any personal identification data that's related to these specific users. There is a consideration of privacy that is made with almost all the data that is collected, but the way that most of the crowdsourced data comes to the transportation agency is that it is stripped away and they're getting raw data. This would be like speed or travel time or different sources of information that don't have any specific type of a personally identifiable data.