OpenHalton

The Open Data Report by Milton city staff is ready for council!

At 2 pages long, it’s crisp, quite balanced, and does take into consideration costs & risks of an open data initiative by the town. Understated is the focus on CITIZEN VALUE of Open Data –  particularly when the minority of those “software developers and tech savy citizens” turn Data into Citizen-Ready Applications used by the majority of residents (web, mobile, etc).

For Milton where median age of residents is 34, an open data initiative can provide a powerful platform to extend and augment the services provided by the town. Think services like looking up a bus route from Milton to Burlington via Oakville Whole Foods, i.e. a Halton Transit web and phone app — one of OpenHalton’s Planned Projects, or perhaps a better way to engage with council via social media provided by apps like WardRep.ca.

The benefits of open data are not ONLY on the transparency and accountability side of things, but ALSO on practical uses by Miltonians in their every day activities: transit, driving, parking, recreational activities, etc. There is still an opportunity for Milton Council to recognize that there is tremendous value in hundreds of applications ready to be used by residents of Milton, if only the data from the Town could were made open data!

Report ends with a recommendation (contingent on council’s approval) to run an Open Data Pilot Project…

…with a limited data set including but not limited to: transit routes, transit stops, ward boundaries, Town facilities and park locations. This will give staff an opportunity to develop a process for posting the data and evaluate the level of public interest…

This is a typical move adopted by almost every municipality that embarked on an Open Data initiative. Particularly exciting is TRANSIT data — as well as Town facilities & Park location information. On the heels of Parks & Facility Data released under the Burlington Open Data Pilot, the Town of Milton could take advantage of apps & visualizations like Burlington Parks and Milton Splash almost immediately!

One area that needs to be challenged in the report is that:

City’s new website, Milton.ca, is not physically able to host some of the open data formats

Really? Which formats? Certainly Burlington, Ottawa, Toronto, Windsor, Region of Waterloo, and many others in Canada do just fine with their existing Content Management Systems (CMS) making the data available. Certainly for  the pilot there’s no technical reason that I see not to use city’s existing infrastructure, particularly after the website re-development. Can anyone point to reasons I’m missing here?

Of particular interest is the reference to other Canadian cities’ open data projects, and specifically likely adoption of the UK open government license — a good move.

The report is well done. Perhaps a closer look at costs of publishing raw data in the existing web/CMS infrastructure and a better articulation of benefits around citizen value?
But overall — not a bad start!!

This tweet from Milton Councillor Rick DiLorenzo has re-ignited possibilities offered by Open Data:

OpenData coming back 2 #Milton Council A&P committee tonite 7pm. Staff recommend add to 2013 budget as pilot project @OpenHalton

The  ADMINISTRATION AND PLANNING COMMITTEE  agenda for April 16, 2012 has this agenda item referencing the Open Data Report (CORS-07-12) prepared by Milton town staff:

CONSENT ITEMS
1 Staff Report CORS-017-12

Subject: Open Data

Staff Recommendation: THAT report CORS-017-12 regarding Council’s inquiry into the costs, resources and risks involved with the possible implementation of Open Data be received for information;
AND THAT staff be directed to include in the 2013 budget a program for Council consideration to enable the delivery of data in an alternative format to support an Open Data initiative.

Bottom line, there is a _possibility_ that 2013 budget may include a program for open data.

Peter Krantz published this article discussing whether Governments really need APIs for open data? Wouldn’t a “download” of a dataset suffice?

My perspective: It depends!

• Is it one or multiple related datasets? Are you looking to expose the relationships between the datasets?
• How large is your dataset & do you have the infrastructure to support downloads of large files?
• If not, do you have skills to support exposing “slices” of data from your internal systems as externally-accessible downloads?
• Are you exposing any value-add functionality of your internal systems (i.e. geo-spatial query on a dataset that involves additional APIs like directions)
• How frequently is this data changing? Every minute, every day, once a year? Think real-time GPS data or environment monitoring vs list of facilities.

Here’s where APIs offer advantage:

• Querying large datasets for relevant bits of data (think a 15GB download vs. a 100KB slice of that same data via an API)
• Real-time or frequent update scenarios (GPS bus tracking, current weather where the data is time sensitive & importing it would be inefficient)
• Exposing relationships in the data (the agency is best suited to expose the relationships in the data it provides via APIs vs just meta-data)
• Using its own APIs (the government uses own APIs for visualizing/interacting with the data or presenting it to citizens as information)

With the right API you can still enable download of the data. Built correctly, an API would be able to handle traffic or bogging down internal systems. This is particularly true for APIs where the data flows from internal systems into a cloud-based open data catalogue, with an API that exposes the data + relationships + downloads. The best of both worlds.

That said, not having an API should not be roadblock to publishing data. Downloads are fine for most common scenarios, but cloud-based API are an evolution toward a more dynamic platform for open gov data.   

p.s. Thanks Jury for a good find & letting us all know!

Open Data in Milton, ON? Not quite yet, possibly not at all!
But why? Things seem to progress so well with the Open Data motion!?

What happened: Councilor R. Di Lorenzo withdrew his motion and asked staff to report back to council on the risks, benefits and costs involving the possible implementation of an Open Data program.

What it means: The motion was shot down by a few concerned council representatives who didn’t want to endorse principles of open data and open standards, or take any further action on open data without town staff first telling them what they think it means from a cost, risk and benefit perspective.

Instead of council directing the town staff to report on practical steps toward open data (as outlined in the motion), the staff was asked to report to council on hypothetical risks/benefits/costs. This essentially relieved  council from endorsing open data, open standards and the principles of open government. While frankly I think it’s a cop out on the part of the council, I also take responsibility for not covering our bases to proactively address the challenges that stalled the motion, outlined below.

How it went down: The motion was read. Councilor Di Lorenzo said a few words to explain the motion making some great references to “planting the seed that will produce results in long term”, as well as indicating that the motion represented “gradual steps” toward open data. Councilor Hamid seconded the motion. Following that I delivered a 10-minute delegation on Open Data and why I thought it makes sense for Milton. After that the floor was open to discussion, which turned into a heated debate.

In retrospect, there were some fundamental challenges with some of the wording of the motion, and also with how I approached my presentation. Those boil down to three issues:

• Some on the town council had no understanding of open data: the definition, the principles, or even the difference between data vs information. This created confusion as to why the motion was put forward, with questions like: “Aren’t we already open? Don’t we already share all this information?!” The opportunity with my delegation was to educate on the fundamentals vs. talking about successes of other cities with open data what’s possible.

• Some of the council were very vocal about costs and risks of moving toward open data. Extreme scenarios from “spending millions on a new IT system” to “hiring a full-time staff to manage open data” took focus off the main action in the motion: to direct town staff to come back with exactly that – an analysis of costs and risks.

• But it was this third challenge that proved to be the biggest obstacle: there was a distinct lack of comfort with the “move as quickly as possible”  wording in the motion below:

Open Standards – the Town of Milton will move as quickly as possible to adopt prevailing open standards for data, documents, maps, and other formats of media

Bottom Line: those challenges could’ve been proactively addressed through engagement with council ahead of the motion, a delegation that focused on fundamentals & education of what open data was, and focused on building consensus behind this motion as a very first of many steps toward open data.

That’s where we are. It’s not all over yet, as the town staff is now asked to investigate open data. The hope is that their analysis will focus on the “low hanging fruit” for open data, targeting a few reasonable datasets and processes that could be easily augmented for open data, and taking a reasonable scope that will make open data for Milton practical vs unattainable. 

Today Milton, ON town council is considering an open data motion put forward jointly by two Milton councilors: Rick Di Lorenzo and Zeeshan Hamid.

The motion specifically calls out Open Government, Open Data and Open Standards as the areas that contribute towards transparency, sharing of information, improving services, efficiency of government services and creating a more economically vibrant community.

This is an opportunity for the town to endorse the following principles:

Open and Accessible Data – the Town of Milton will freely share with citizens,
businesses and other jurisdictions the greatest amount of data possible while respecting privacy and security concerns; 
 
Open Standards – the Town of Milton will move as quickly as possible to adopt
prevailing open standards for data, documents, maps, and other formats of media;

and to implement specific actions for the city staff to report back to council what steps could be taken on existing data, with the relevant focus on costs, risks and benefits. 

This motion is modeled after similar motions – like those by the Cities of Vancouver, Edmonton and Ottawa, and many others that paved the path for open data initiatives in Canada.

I will be representing (and tweeting from) @OpenHalton, as well as deliver a delegation on Open Data with the slides posted below:

Happy 2012! Things have been busy on the Open Data front in the Halton Region – with Burlington, ON open data pilot announced and launched in the fall, and even some apps & visualizations of Parks Data built by open data hackers like Matt Down. Right before the holidays, I also heard from the Milton councilor Rick Di Lorenzo about his plans to draft an Open Data motion to put in front of council in early 2012. Good stuff!

However, not much is still happening with the Halton Region, despite a variety of great data already available. A good example is the Flu Clinic information, including dates and locations, published on the Halton.ca site (as an HTML list and PDF). While there was clearly a lot of effort put into creating the web page listing, and into creating a nicely-formatted PDF, it is a shame that the source data was not released in a spreadsheet format. If it were open data, it would be trivial to create and share map all of the clinics and not just links to each location.

The Halton Flu app is intended to be exactly that kind of map. It’s just a simple site for web & mobile devices, with a clinic list on the left, a filter by city, search, plus an interactive map on the right – in a map layout & format that most folks are used to.

The objective: making it easier for residents to find nearby Flu Clinics in Halton. But there’s more!! Why not then add a way for users to geo-locate themselves on the map, and a way to get directions to the clinic of your choice? Done, and done!

The credit for the work on the site goes out to Johan Säll Larsson, a developer behind the JQuery-bing-maps framework, who saw my attempts to use his code on the Milton Splash project and offered help to add some dramatic (and much needed) improvements. Johan and I worked together to liberate & scrape Halton’s Flu data – into a developer-friendly dataset and made it citizen-ready via our Flu Clinics app.

The code is open source (contact me if you need the code right away), and could be used and re-used over and over for other similar application needs in Halton & our municipalities: think finding Battery Recycling centers, Hazardous Waste, Tire, Electronics disposal, etc.

My hope is that Halton, and in the very near future Burlington, Milton & others will start releasing open data that could power similar citizen-ready applications.

With the Burlington Open Data pilot in full swing, the very first piece of data in the Halton Region released under an Open Data license became the “long hanging fruit” dataset listing Burlington Parks / Facilities / Fields and Courts.

The dataset released on Sep 19th was released as a spreadsheet with multiple tabs – and formatted to make it easy for citizens to consume with spreadsheet software.

Within a few hours, I built a Burlington Parks Finder app (web & smartphone mobile), which was a quick adaptation of the Vancouver Parks Finder, previously built using Vancouver’s park listing from their open data catalogue. As with Vancouver’s data, the application leverages the SIMILE Exhibit software, which makes it easy to build map / search applications even for those with limited developer skillz (read: that’s me )

This is open data in action – an example where cities can leverage each other’s open data initiatives to release (and improve upon) datasets, which are similar in formats, schemas and range of information. This makes it extremely easy to customize and re-use in similar type applications – saving time for developers, and providing immediate value to citizens.

City of Burlington just launched an Open Data pilot, proudly joining the likes of Vancouver, Nanaimo, Toronto, Edmonton and Ottawa. Championed by the city’s Information Technology Services department, this initiative is a solid step towards improving transparency, accountability and citizen engagement.

Earlier in July OpenHalton was involved in helping organize an “Open Data e-Gov Focus Group” and that same month I was invited to speak to council in support of on Open Data initiative spearheaded and presented by Christine Swenor, Director of IT Services. The council was very receptive to the initiative (the full webcast recording is here). Within just 2 months Burlington was able to launch a full pilot, with the following objectives outlined in this memo to council:

• The goal is to better understand all aspects of open data, including resource requirements and benefits, which will better inform the e-Government Strategy;
• The pilot is being supported by the Parks & Recreation, Clerks, Legal & IT Services dept’s
• Parks & related facilities data will be published as data sets for the pilot
• Other datasets (!) may be added over the duration of the pilot where appropriate and manageable.

Of particular interest is the following quote from the memo:

It has become evident that Open Data is a key component of Open Government and should be addressed within the e-Government Strategy.

Indeed, this is the type of a holistic view that many other cities could benefit from, as they look to refresh their websites or update their citizen services online. Burlington seems to be quite serious about the role of open data in driving better citizen services:

The purpose of Open Data is to enhance transparency and accountability and potentially service delivery.

Well done, Burlington, for recognizing the potential offered by Open Data, and also for championing the movement in the Halton Region!

Critics may point to the Terms of Use issues stemming from re-use of Vancouver License, or that there’s just one dataset (schema seems to be influenced by this Vancouver’s parks listing) or that it contains just point-coordinate data vs. complete park boundaries. However, those are part of a learning process a city is expected to go through as it matures its understanding of open data and refines its strategies for open government.

As I look at the flurry of activities just west in Hamilton, led by our friends at Open Hamilton, I can’t help but think we’re starting to get somewhere with this Open Data thing…. Now we just need to get cranking on building apps from this data to show what’s possible

[image from Flickr: http://www.flickr.com/photos/batara/4482071468/ ]

If you’re a government agency evaluating ways to get started with your Gov 2.0 / Open Gov / Open Data initiatives, keep in mind these 3 simple strategies for a quick win:

1. Review the terms of use

Even if your agency doesn’t have an Open Data policy, your agency’s website could have potentially restrictive terms of use policies. When looking at enabling Government as a platform, a quick win is to review and revise your site’s terms of use.

Specifically, are you explicitly preventing someone from using or even linking to your site’s information? Citizens that want to leverage and re-use public information on your site — for example, waste pickup schedules, council information or ward boundaries — may be legally bound from doing so. Ensure that this type of information isn’t restricted by “sweeping” terms of use policies, it can be as simple as revising the footer of those web pages. For more see this piece on licenses at Eaves.ca

2. Publish the original files

Another quick win is publishing the “raw” structured file that were originally used to create the public information on your agency’s site. More often than not, the print-ready documents in formats like PDF originate from machine-readable, structured documents or spreadsheets. While some argue that PDF does a good job of “preserving document integrity”, it often handicaps efforts to automatically extract the data.

If your web pages or PDF downloads originate from a spreadsheet, document, or any other type of a structured file format (including geospatial formats) — offering up the raw files saves developers the headache of reverse-engineering the documents you can just as easily publish online, along with the PDFs if you so choose.

3. Make open what’s already public

The last tactic is identifying the “low hanging fruit” for open data — typically information that’s already public. My favorite are various geospatial datasets that you may already be sharing today via maps, guides, etc. If your agency is using GIS (Geographic Information System) software, you can simply export the data that was originally used to create those nice citizen-friendly guides & community maps into a popular format like KML. Think maps of your agency’s facilities, points of interest, parks, city and ward boundaries, etc. This GIS data can then augment the static maps and power some very useful citizen-ready Gov 2.0 applications.

These are some simple but effective strategies to get the ball rolling with your Gov 2.0 initiatives.

a Case Study for Municipal Open Data

The heat is scorching! Residents across Ontario, Quebec and part of the US are trying to stay cool. Many seek out public swimming pools and splash-pads, and turn to their municipalities for information. Others, like Joey Coleman of OpenHamilton and yours truly seek out ways to make that information more accessible.

Hamilton’s Dowsing and Milton Splash are two of the most recent examples of what is possible with open data. They represent a a real-life case study of how Open Data can help keep us cooler, while also helping cities provide a better service at a lower cost.

This is how:

STAY CURRENT: Many municipalities – like the town of Milton – provide great-looking printed community maps with swimming pool & spray pad information, and various community services guides. The challenge with those are production and printing costs, which pose a barrier to keeping the information current. For instance, the Milton map is missing some of the newer facilities (like the 2 splash pads in newer areas of Milton).

The solution: post the source data for the map – i.e. a machine-readable list of facilities with geographic coordinates. No fancy formatting, no map production, or printing, or distribution required – save our taxpayer’s dollars. Just publish the raw, most up-to-date data online, the data that already exists in town’s information systems. To Milton’s credit, town staff produced a Beat the Heat poster with an updated list of facilities, which even included an advisory on one of the spray pads closed for repairs. The obvious challenge: what happens when repairs are completed, but the flyer is still in circulation? Again, open data to the rescue:

ONE SOURCE: With so many sources of information (maps, guides, flyers, website pages), open data can become one definitive source of data. OpenHamilton’s Dowsing does just that by pulling partial data for water facilities from at least 4 sources into one dataset. Milton Splash similarly integrates information from 3 printed publications into a single dataset. One place as one definitive source of data drives better accuracy and also better government efficiency: cities with open data catalogues discover that not only citizens, but also city staff use those catalogues as the primary source of data.

When implemented correctly, i.e. with workflows and processes to keep the data current, cities can realize significant savings by having just one place to update. Many open data catalogues, such as Microsoft’s open source OGDI or commercial Socrata provide open standard Application Programming Interfaces (APIs) to the data. This creates a cascading effect, with open data API’s driving many different uses:

MANY USES: Even without “fancy” APIs or “catalogues” any municipality can realize the benefits of open data. All it takes is data, a website and a license (a license outlines the terms of use / agreement for how the data should be used; for more info on the topic of licenses see this recent article on the state of open data licenses in Canada).

Once the data is online in a machine-readable format, it can literally “turn on” any number of web pages, digital maps, visualizations, online reports, web and mobile applications, and even ordinary spreadsheets like Excel, accessible for those without programming skills. That is Government as a Platform: the vision popularized by Tim O’Reilly. For water facilities all the cities need to do is provide the names, coordinates, hours & status (advisories, etc.) as a download. For Milton I used OGDI allowing me to make the data accessible as an html table, or a file download (CSV, etc), a map or KML download (common mapping format), or as an XML oData feed or JSON API to power any number of interactive maps and apps just like Milton Splash .

Dowsing and Milton Splash are just small examples of what can be accomplished with open data. Both are relatively uncomplicated apps, but each can provide a useful service to Hamilton and Milton residents searching for a pool or a spray pad nearby. As the heat wave breaks records, one couldn’t ask for a better way to showcase the value of Open Data.