Northernlands 2 - Do cities have access to the private sector data they need?
Jack Hardinges, who leads on the 'data institutions' progamme at the ODI, explores the current state of data access in cities
This transcript comes from the captions associated with the video above. It is "as spoken".
Hi everyone, my name is Jack Hardinges and I'm program lead at
the Open Data Institute. I'm really pleased to be pre-recording this
for Northernlands 2020 and looking forward to joining lots
of the other sessions around this one next week.
In this session I'm going to be setting up two arguments in
response to the question of do cities have access to the
private sector data they need?
In terms of an agenda I'll first give a bit of context to the question so
where it's come from and some of the ODI's work on this so far and
I'm going to present the case for yes. Cities do have access
to the data they need, which will feature Richard
Emmott at Yorkshire Water, and then I follow up with some
evidence in favor of no, they don't. So we will be hearing
from Marius Jennings at Bristol City Council as well. And then I
finish with some ideas about what might come next, and open
up for some live Q&A.
I'll start by saying that cities have long built and maintained
infrastructure. We depend on their roads and bridges such as
these nice ones belonging to Hong Kong, as well as things
like their communications networks and utilities.
And given that I'm speaking at a part ODI Leeds event I perhaps
don't need to make the case that
cities are an important lens to use to talk about infrastructure
and responsibility is about building and maintaining it.
At the ODI we talk about data infrastructure, so to
us data infrastructure, includes the data itself.
So statistics, maps, sensor readings as well as the
people, processes and technologies that enable it
to be used. We think that data infrastructure will only
become more important as our economies and our societies
become ever more reliant on getting value from data and
from a cities context as more and more people end up
living in urban environments.
As is similar with physical infrastructure, almost
everywhere a cities data infrastructure consists of data
that's managed by public institutions, so housing
authorities and transport authorities as well as data that's
collected and held by the private sector organizations
that operate within them. So when we think about private
sector city data, we talk about both data generated by
businesses that work with cities to deliver public services as
well as those that provide services directly to their
citizens, visitors and others.
I guess one of the underlying arguments for lots of our work
on this and related topics is that we strategically plan fund
and govern our physical infrastructure to varying
degrees of success. And but we're not necessarily treating
or thinking about data in the same way right now.
So last year we set up small research projects as part of our
R&D program to explore the current state of city access to
private sector data, and we set out to find out three things
really. One of which was do cities have access to data held
by private sector organizations, and how do they access that? So
what types of processes in what kinds of ways? And then also
what are the challenges to cities doing this? So what are the
barriers? If we are right, but we're not necessarily thinking
about city data infrastructure in the same way as physical
infrastructure. I should say that when I'm talking bout
cities in this context, I'm using it as shorthand for city
government and public sector bodies as well as local
organizations, communities and people. So it could be
researchers startups. So when I say cities I'm meaning something
quite broad, deliberately here.
As you might guess we would, we tried to do some of this
research in the open, so this is a open document that we set up
for people to contribute to and use as we went. Includes some
really interesting examples of cities accessing data held by
the private sector and what we've done is
set up this matrix, which kind of clusters them around the
purpose of the use. So is it to support innovation in a
particular place, or is it more about transparency and
accountability? We've also set out the role of cities or the
approaches they used to access that data. So are they collaborating
with the data holders? The firms that have collected it?
Or are they having to in some cases, legislate for access to it
So it's a real mix of different approaches in different examples.
Been really pleased by the contributions from the
community on this - it's been quite popular open document, so if
you've added to it and are listening, thank you very much.
Say onto the question and the main body of the session today.
So do cities have access to the private sector data they need?
I'm going to be using examples from the open document I just
shared to present the case for both yes and no. In response to
this question, and I say it now rather than leave it to the end.
But obviously the answer is it depends, and in particular it
depends on what data you're talking about and what city and
bear with me while I present these binary polar options, it's
deliberate and I hope it's a
useful way to surface some interesting and concrete
examples of what's happening or not across different cities.
Let's start with yes.
So firstly we came across some really interesting examples of
companies producing or commissioning research about their
impact in particular places. So this is an extract from one of
Airbnb's Insight reports related to the West Midlands, so I think
this is evidence of recognition from some organizations the
data they hold and collect about a particular place has
value to it should be accessible to people and organizations
other than just themselves. This is my evidence point for that.
Alternatively, cities can turn to some of the more
specialist data providers for access to data about things like
the movement of people. So this is a foot-fall graph from a
company called Safe Graph who generate these through access to
cell phone activity data. This might be particularly useful to
a city looking at the impact of COVID-19 on movement and the
local economy. Separately, there's some cool examples of
cities working with the private sector to collaboratively
maintain and produce data, and based on our research in
particular, lots of maps. So this is the Greater Manchester
Open data infrastructure map, which includes data from the
private sector. So it has things like cycle network data and the
layer that I've got here is Zoopla property data.
Another example is this one from Amsterdam, so this is an attempt
by the City Council to build a sensor registry that
encourages both government departments and companies to
upload the location of cameras and sensors in that
particular city mainly for transparency purposes so people
know when data about them is being collected and where.
And lastly, on the case for Yes - we came across some quite
sophisticated attempts by companies, particularly in the
transport sector, to develop services and tools that provide
quite enriched access to data that their services have
collected. So this is a screenshot of Stava's Metro
service that enables city planners to access data to
better understand trends in movement and to inform their
planning and decisions about investment in transport
infrastructure. I just chose this example from Louisiana as I
like the name of the place.
And just to say, this isn't always about static data and
maps for policy policy making and planning. This is a
picture of Transport for London's operations room and
the example that we came across was it now ingesting
real time data from the mapping service Ways which it uses for
operational traffic management.
And I'll cut to Richard for his view on this.
My name is Richard Emmott I'm director of corporate affairs
for Yorkshire Water, and I've been leading our open data
policy since 2018. I think there is a lot of potential for more
data sharing from the private sector within cities, although I
think it's come a long way, particularly in the fields of
transport. From a Yorkshire Water perspective key areas of data
that we've been looking to share with other public service
providers have been things like information on flooding, how I
St works are operating, and other issues around how we
employ people such as gender diversity and how the workforce
of our organization is
constituted. The way we see that as a possibility is that it
means that the data sharing process can enable us to
develop really strong joint partnerships with our public
service provider partners to improve service within
the cities and to provide a much more integrated data landscape
for the city.
So the first stage is what we've done to share data at Yorkshire
Water is to participate in Data Mill North, and we published a
whole range of data sets on their in fairly raw format.
But obviously that's quite static data albeit it that it
has made our information accessible alongside some of
the key public service partners. The next stage of that which
we're about to launch shortly is to run a series of consumer
customizable dashboards, so that's what you can do is kind
of take out our raw data feeds from the website.
And drill down into really quite local granular level to
understand and analyse our performance on some of the key
issues within really quite quite small individual
communities. We think that's a really exciting possibility and
it starts to kind of create a different relationship with the
public in terms of accountability and transparency.
So we're really keen to get that launched and see what sort of
public response we get to it.
There are three real benefits for Yorkshire Water ourselves,
and that's transparency, accountability, and potentially
innovation. In terms of transparency, we are, we provide
an essential public service and the public have a genuine
interest in finding out how we're performing. So it's
important that we completely open about both the good and the
bad of our performance. That's why, for example, we published
five years data worth on air pollution performance, some of
which doesn't make for a great reading. It's important that
people understand that.
And that leads obviously to accountability. I think where we
have to do more work is around how we can collaborate with
other partners, potential service providers, on how they
can use our data to innovate and give us some different
approaches to things like leakage or flooding.
In terms of partnership with local authorities I've always been
very conscious that taking on its own our data is useful, but
it's much more useful when you put it alongside of data, say,
from the Environment Agency.
So if you got some of our data on drainage planning and
flooding, put that alongside some of the Environment Agency's
data, say on river levels and potentially get a much more
dynamic and rich data set that there's a hell of a lot more use
to potential data users. I think most organisations see the
barriers to data sharing as being in two sectors.
One, there's a big concern about cyber security and data
integrity and the whole GDPR
regime has been a challenge and certainly that was that came in
at the very early days when we were starting down our open data
journey. Likewise, cyber security. Clearly there's also a
commercial sensitivity around access to some data, and there's
also a general nervousness I think about opening the inner
workings of an organization out up to the outside world. I think
what we found when we've started doing things is that
to some extent, we've had to be fairly cavalier about some of
the rules except for GDPR which we've respected very
carefully. But I remember a conversation about potential
cyber security data and some of our water treatment works, and I
discovered that the information that were not supposed to
release was actually already publicly available on Google, so
it was particularly pointless to sort of maintain that
restriction. So I think I'd recommend the approach that we
took, which was to publish some data, see what happens, if the
sky doesn't fall in and you find something interesting out as a
result of it publish some more. Build your sort of risk
tolerance as that happens in your skill and expertise in
doing it. And also kind of build the richness of your
engagement with potential data users along the way. And
gradually they'll start to educate the organization that
this is actually a very powerful and interesting thing
to do, rather than something that is challenging or difficult.
And now onto the fun part the argument for No.
And to start with and I don't have a screenshot for this, but
these are two quotes from our research that we found
interesting. That kind of show the extent of the hacks and work
arounds that cities are having to put in place just to try to
access data and then to use it to inform their decision making.
So the first involves pinging Uber and Lyft's APIs, which is
interesting as a form of data collection, to say the least.
I'm not sure how robust that is and the 2nd is an attempt to
essentially replicate the datasets that these kinds of
rideshare companies already have and to me creating that kind of
shadow data set feels like fairly fairly unnecessary
effort that shouldn't necessarily need to be expanded.
Again, no screenshot, but a quote. This is another that we came
across that is stuck with us. This is a description of a
particular company's view of
Copenhagen's data platform and describing it as parasitic and of
no real benefit to the company's the city would like to
contribute data to it has stuck with us and some of our further
research has really validated this. In particular a piece of work
that we did in 2018 which found that as a city or borough in the
UK it was nigh on impossible to find out how many properties
were available on Airbnb or short term letting sites in a
particular area. Essentially, the data infrastructure didn't
exist. It wasn't the case that it was broken, it just wasn't
there. And this screenshot looks great, but it's not, and I'll
explain why. So, because the short term letting platforms
don't share data with anyone robustly, or if they do, it's
very limited research purposes. This is a service that
sprung up called AirDNA, that systematically scrapes these
platforms and packages up the data like this so you can derive
different kinds of stats and analysis from it.
But not only is this on fairly dodgy ground from a
licensing point of view in terms of whether or not the
data can be scraped from the platforms in this way. If you
come to any conclusions based on this data then the
platforms tend to dispute it, because they argue that
the data is inherently inaccurate and doesn't match up
with their own internal records. So there's
real limitations to what you can do with this.
Lastly, on the argument for, no, cities don't have access to the
private space they need. I point to the various live battles in
court that we came across in our research, especially in the US
between organizations like Uber and Lyft and city transport
authorities over access to data as evidence of it not being there,
that infrastructure not being developed or governed in the way
that we need it to be.
So since 2017 New York has made access to data, part of its
licensing conditions. So in order to operate as that
particular kind of operator, then you need to provide monthly
access to data that you've collected. And others seems to be
following suit, and there's some controversy over whether or not
some of these attempts by cities constitute over-reach and the
privacy implications of those.
And as well as cities, drivers on these platforms are taking legal
action to access data that these types of platforms hold. So this
is a screenshot of James Farrar's data, who's an ex Uber
driver and is taking them to court in part over fair access to
data that the platform holds about him in order for him to
build a case or at least understand his employment and
the way that he was directed by the platform over a
particular period of time.
Now I'll cut to Marius.
My name is Marius Jennings and I work for Bristol City Council as
our urban data lead. So my work is really seeing how we can work
with internal partners and
external partners to make open data are available to support
our understanding, our city challenges and to promote open
data in general.
I don't feel that cities currently have enough access to
private data to tackle the range of challenges they face, and I
think that's particularly evident at the moment. We are
facing some of the most momentous change ever. And
some of the most ambiguous situations that cities of had to
cope with. And this is all happened really, really quickly.
And this means that we're going to have to start doing things
in different ways. We're going to have to start looking at our
macro environment and
seeing how do we experiment and what do we need?
And that's a real challenge, because a) it's identifying
what is the information that we need internally and
then b) how do we actually go about getting that data
that may not be available?
So there are a couple of initiatives that Bristol City
Council are exploring.
One of them came out of the ODI. open cities workshop that we
held in December 2019 and what that really gave was an
opportunity for us to bring together a range of
stakeholders to see what data is available and really map our
ecosystem. And understand what are the ethical considerations
when you're getting data from different parties and how do you
share that information
Understanding what context is it suitable to share that data?
So some of that data on the data spectrum may be freely available
that we would want to share with anybody. Some of that might,
however, be quite confidential and not appropriate. And how do
you best go about it?
In a legally compliant way where you are seen as a
trusted provider. One of the things that we're looking at in
Bristol City Council is working with our procurement team so
that as new contracts are put into place, we look at social
value, and ask private sector companies to provide data that
is not necessarily
commercially sensitive with the council to kind of help shape
our understanding of our city
and provide that data meaningful ways with our citizens.
And another initiative that we are involved in is called the
Bristol one city plan, and it's the city's aspirations for where
we want to be by 2050, and what that really does is it moves
from the council taking ownership for those goals to
rather a collective co-design piece where we work with city
stakeholders to make the city a
fairer place while
making it a cleaner, more economically viable city, and
what that means is that we're having various KPIs that we
need to look at on a yearly
basis. And working with our city partners so that we can make
that information available to monitor how we're progressing.
So unfortunately there are a number of limitations around
sharing data. In terms of.
The private sector sharing and local council. So that it can
be used in an effective manner and drive the best value.
So some of those things are a) Is there a business case to do
this? At the end of the day often councils don't necessarily
have the data science or the GIS or business intelligence teams
who can devote the time to
take data that's given and really extrapolate the value.
It often means that there isn't the understanding from private
sector organisations on the sort of data that you're asking for,
and that may mean that there's quite a low risk appetite where
they're not necessarily engaged because they're not seeing the
value of that.
They may be user agreements that are in place that hamper
the availability of data.
They may be
potential commercial sensitivities around the data.
So there are a number of things that have to be taken
into account and addressed before you can do this in an
So I do have a number of recommendations for improving
the situation, and particularly about how we get to work more
effectively with private sector
organisations. I think one of the main things is education.
It's been going out there and really talking about the benefit
of providing that data.
Easing the private sector organisations' concerns
and showing that often
customers are really interested in knowing
how they're doing and
if they are having that interest and that honesty and
transparency, that's often quite
well received. And though they may not be performing as well on
some of the issues, it doesn't mean that they're going to be
criticised for that. If they do it in quite a transparent way.
I think there also needs to be a better understanding with
organisations and also councils around GDPR. Often that gets
utilised as an excuse.
And it doesn't necessarily need to be, particularly if you're
looking at more open data, or the data is being shared in a
compliant way that is taking
into account concerns.
I think there are also circumstances where councils
will automatically have powers to access certain data,
like if there was a major incident. That often means that
different organizations come
together. To enable the city's civil contingency plans. So some
of this work does happen in the background, but people may not
be aware of it.
I probably would also say tackle one thing at a time.
It's complex. And if you have already got existing
relationships with private sector organizations, spend the
time with them, understand what
their doing. For example, we work quite closely with Bristol
Waste. To be fair, which is a Bristol City Council owned
company but really what they do is share how they're performing
and how they are able to
recycle more each year and that enables us to understand
how Bristol responds to the climate emergency, where we're
trying to be carbon neutral by 2030. I think if they can be
seen mutual goals being aligned by this data sharing people
might be more engaged to do it.
Also, I think you need to have quite an honest discussion
internally and review what capacity you've got to utilize
this data because there's no point in going out there and
getting it and then finding that you're not actually able to
interpret effectively or share effectively. There needs to be
support the local council's giving to the
external organization so that that data is used in
the most effective way.
And so to wrap up what next? And from my point of view, the real
conclusion from our work so far is that while there are some
really promising examples of open collaborative access to
data between cities in the private sector, and this isn't
spread evenly and there remains a lot more to be done.
This is actually the first time we presented some of our
findings from this research, and so ideally we'd like to go
further. And if it's useful to produce some guidance for cities
or for companies thinking about this issue based on the things
that we've come across and what might constitute best practice.
Even better would be a pilot or two or some practical work in
this topic. So if you are one of those two groups, then do
let me know if you're interested in talking to us and potentially
working with us. For cities we also run an online workshop
that came out of the project
that I described. Where we use tools such as our data
ecosystem mapping tool and the data ethics canvas to help
cities understand current gaps in their data infrastructure or
to identify the ethical implications of increased data
access or a new data project.
And for private sector data holders, we actually have a
series of webinars at the moment that are working through similar
topics to the one that I've covered today. So do you have a
look on our website for those if
they're of interest? And I'll stop there to take any questions
you have or ideas you wanted to share. Thank you for listening.
Programme Lead - Data Institutions
The Open Data Institute
Jack is Programme Lead for Data Institutions at the ODI. He is responsible for the strategic direction of the ODI’s work to explore different approaches to data stewardship, leading the delivery of related projects and building partnerships with other organisations.
Recently, his work has focused on involving the public, patients and other stakeholders in the stewardship of health data, and the flows of data between cities and private sector organisations.
Nothernlands 2 is a collaboration between ODI Leeds and The Kingdom of the Netherlands, the start of activity to create, support, and amplify the cultural links between The Netherlands and the North of England. It is with their generous and vigourous support, and the support of other energetic organisations, that Northernlands can be delivered.