Map Of The Week: Urban Reviewer

Welcome Eric Brelsford to our Map of the Week series. Brelsford is a freelance programmer, mapmaker, and teacher. He spends much of his time working on 596 Acres and related sites in Brooklyn, NY.

Brelsford, as part of 596 Acres, has joined forces with Partner & Partners and SmartSign to create an online map showing over 150 master plans for neighborhoods adopted by the City of New York. The process covered the follow up on a Freedom of Information Law request for the records and the translation of paper plans into machine-readable spreadsheets. The result, after two years of work, is urbanreviewer.org a map where you can take a look at which areas in New York have been affected and what those master plans were.


About Urban Reviewer

Master plans have been created for many cities in the US since the mid-twentieth century. The plans were often adopted by cities to receive federal funding that would facilitate redeveloping so-called “blighted” neighborhoods. This involved using eminent domain to acquire the land, relocating the people who lived there, and demolishing the structures on that land. Urban Reviewer maps New York City’s master plans that have been created since the 1950s.

Urban Reviewer, zoomed out

These plans significantly changed our city’s neighborhoods. Whether the plans for new industrial or residential space made it to completion -and many did not- they still involved destroying structures and moving communities to other parts of the city. Part of our motivation for creating this map was that we wanted to see which pieces of land (“lots”) were affected by master plans and are still vacant. Our map’s filters enable this now:

Urban Reviewer, current day vacant lots highlighted

Data collection

Many volunteers gave a total of hundreds of hours in order to digitize paper records of the master plans provided by a city agency. These detailed records included information about the plans and each individual lot within them and are the definitive records of these plans. We are not aware of digital records of the plans, so we created them.

There was a moderate amount of processing that had to be done in order to make these records useful, but we will stick to the CartoDB side of this mapping here.

Inserting the data

Once the data is ready to be imported into CartoDB, we insert it into our tables using the SQL API through cartodb-python.

We use two CartoDB tables:

  • plans, which stores each plan’s name and other details, and
  • lots, which stores the geometries for each lot and points back to the plans table.

Viewing and styling the data

Urban Reviewer uses a Leaflet map with a Mapbox base layer. The CartoDB layer of all the lots is added using CartoDB.js.

We style the lots using some pretty basic CartoCSS that makes the lines around the lots thicker when zoomed out:

Interactivity

The master plans can be complicated. They often consist of numerous lots that are rarely contiguous. It can be hard to tell if a lot is part of a plan, and some plans overlap. We rely on interactivity to make these relationships clearer, and we’re still tweaking how this works.

Adding plan boundaries on hover

First of all, it’s helpful to be able to see the boundaries of a plan while moving your mouse over the map and while a plan is selected:

Urban Reviewer, hovering over a plan

This is one area where CartoDB’s SQL API really shines. Traditionally you would need to write code on a server that would collect the relevant geometries and process them. With the SQL API we could write a few lines of JavaScript that ask CartoDB for exactly what we needed. We created a PostGIS SQL query that selects all the lots in a plan, combines their shapes, finds the convex hull of this shape, then buffers that shape a bit to add some space between the lots and the boundary line.

After testing the query in CartoDB’s SQL console, we added code to get the results as a GeoJSON file and add them to a Leaflet GeoJSON layer in just a few more lines:

Highlighting lots in a plan when it’s selected

When a plan is selected we also highlight the lots within that plan:

Urban Reviewer, plan selected

We accomplish this by setting the layer’s CartoCSS using the CartoCSS above and adding a little more:

The slightly tricky part is that in our JavaScript code, before we set the layer’s CartoCSS, we dynamically replace PLAN_NAME with the name of the plan that was selected. You can see more details of how we do this in the code if that’s your thing.

Highlighting a lot on hover

Finally, when you mouse over a lot (either in the list view or on the map) the lot is highlighted in a similar way to how we highlight the plans. First we grab the lot’s geometry with some SQL:

And then we get the geometry using the SQL API and add the GeoJSON to a Leaflet layer. This is nice because we can overlay an outline around the lot without updating the layer’s CartoCSS. That would force the entire layer to refresh just to add a line around one shape!

Now that you know a bit about how the map works, come explore NYC’s master plans on Urban Reviewer.


See our other Maps of the Week, and go to CartoDB to start creating your own maps.

CartoDB at the INSPIRE Conference, helping public sector organizations share geo data

On june 16th, CartoDB’s CEO Javier de la Torre took part in The European Public Sector Information (ePSI) platform workshop at the INSPIRE Conference held in Aalborg, Denmark.

The ePSI platform workshop brought together important industry players to explore the impact of Public Sector Information reuse in their geo-related fields. The presentation given by Javier de la Torre, titled Making the most out of geo information. Next generation data products, showed how CartoDB really change the way people do visualizations on the web, and the challenges of public administrations sharing geographic data and enabling its reuse by organizations and the general public they serve.

Check out how CartoDB can help organizations in the public sector to create and share maps, and work and manage with geo data.

Submit your maps to the FOSS4G Map Gallery

The annual FOSS4G conference is the largest global gathering focused on open source geospatial software. This year will happen in September in Portland, Oregon (USA), and CartoDB will be present with several sessions (How to tell stories and engage an audience with maps, Dynamic mapping on the web: building a scalable service for thousands of companies, The unrelenting progress of design in open source…).

As a geospatial conference, maps will be an integral part of it, and the organization has opened a Call for Maps:

We invite contributions to the FOSS4G Map Gallery to build an exciting and vibrant exhibit of the very best in mapping. This Map Gallery will not only demonstrate the work of delegates but also act as a reference point for the current state of mapping more widely. We encourage all delegates to share one example of their work and help us create a cartographic record of the FOSS4G 2014 conference.

So, we invite all CartoDB users and map creators everywhere to contribute to what will surely be an amazing Map Gallery. Sumit yours today, and spread the word!

And if you need inspiration, take a look at CartoDB’s map gallery.

CartoDB Twitter maps in Today, Yahoo, ESPN, Time, CNBC, USA Today, Daily News, Huffington Post…

During the last months CartoDB maps have made it to lots of media outlets, generating millions and millions of map views all around the world covering very different issues. Some of the maps are about the most global event of the summer, the Brazil World Cup.

Mapping Twitter activity

Twitter is doing an amazing work of analyzing how people is tweeting about the World Cup. You can check out the Twitter Data account. One of the tools they are using to communicate how the event develops in their platform is CartoDB. They load tweets extracted from their platform into CartoDB to generate animated storytelling maps like this one, which illustrates the match between USA and Portugal:

These maps have been getting lots of attention and have been featured in NBC’s Today (one the longest-running daily US shows), ESPN, and publications like Time, USA Today, UK’s Daily News…

Other examples of recent major events include #bringbackourgirls, the release of the new Beyoncé album, the Superbowl final, Real Madrid’s Champions League victory

Maps as engaging content pieces in media

CartoDB enables any newsroom, brand, or agency to create engaging pieces of content in the form of interactive maps that illustrates stories. These maps provide a new dimension to content, giving insights of how people react to brands and events.

Up until know measuring how people reacted to events was a slow and expensive process, but in the social media era everything changes, and CartoDB lets you visualize, analyze and understand user behavior in real time, getting data directly from sources like Twitter.

We are (even) Big in Japan

Also, Twitter maps about the World Cup, made with CartoDB, have been shown in four out of six nationwide TV networks (NHK, Nippon TV, TBS, FUJI TV).

Maps for your brand or event

Head over to our Twitter Maps solution page to get more information and apply for the beta program.

Announcing Twitter Maps, access Twitter data directly from CartoDB

We are happy to announce our new Twitter Maps solution, which lets you consume Twitter data (geotagged tweets) directly from the CartoDB interface. We have integrated CartoDB directly with Twitter, so you can consume Twitter data very easily.

With this new development, you will be able to create animated maps of Twitter activity about your brand or event (or any other thing happening in Twitter you can think of!). We are one of the first mapping tools to provide this service.

You may have seen some of the animated Twitter maps Twitter themselves have been creating lately. World Cup matches, Ramadan around the world, India Elections, the release of the new Beyoncé album, the UEFA Champions League final, #bringbackourgirls

These maps make for informative and engaging pieces, generating lots of map views, social shares, and new insights about the content, further communicating your event’s or brand’s relevance. They have been featured in more than 100 media outlets, such as Time, CNBC, USA Today, Huffinton Post, The Guardian, Daily Mail, Le Monde, Aftonbladet, SZ.de…

Now, everyone can make Twitter maps, retrieving tweet data directly from Twitter from inside the CartoDB interface. Search for your term or hashtag, filter data as desired, and visualize them on a map. From there, use the standard CartoDB tools to create your visualizations and maps.

Sign up for the beta

For the time being this is in beta. Head over to our Twitter Maps solution page to get more information and apply for the beta program.

Altran Spain becomes the newest CartoDB Partner

CartoDB is happy to welcome Altran Spain as their new partner. The company is providing technology and engineering consultancy services for more than 20 years. They are adding CartoDB in their portfolio of technologies with which they develop projects for clients in every sector.

Altran Mobile Solutions is one of the leading expertise of Altran Spain specialized in the creation of global services that allow to collect, analyze and use the data generated by sensors in mobile devices as well as in the development of mobile applications.

Altran Spain has decided to use CartoDB after being interested in the product and the cloud platform we have built, which will let them integrate geospatial and visualization functionality in their projects with ease, quick integration and deploy times, and without worrying about scalability issues. All this while enjoying the possibility of completely managed on-premise service for corporate clients with such policy requirements.

CartoDB has grown a lot during the last two years, hence the number of clients wanting to develop with CartoDB (such as Twitter, NASA or United Nations) and the number of new CartoDB Partners is growing accordingly. We are totally focused in strengthening our current network of partners to help absorb the demand of CartoDB powered projects, and Altran Spain is a terrific addition to the team.

We are totally focused in strengthening our current network of partners to help absorb the demand of CartoDB powered projects, and Altran is a terrific addition to the team.

What entails being a Partner

Being a CartoDB Partner entails benefits, among others, such as:

  • Client and project leads (many companies come to us asking for custom development of CartoDB projects, which we forward to our partners).
  • Special support channels and specific partner oriented educational materials.
  • CartoDB resources so you can try features and develop test projects.
  • Discounts for the reselling of CartoDB subscription accounts.

Check out our CartoDB Partners page for more information and the list of current partners.

Apply to being a partner

Interested in being a CartoDB Partner and develop projects with the best geospatial technology available? Apply today!. You can also check out our Case Studies section to get to know better which type of projects can be developed using CartoDB.

Introducing our Map Gallery

Map gallery

One of the things we are most proud of here in CartoDB is the great number of different maps and visualizations people are creating using our technology. CartoDB came to life because we had our own needs in terms on data visualization on maps; project after project we kept iterating and adding new possibilities, until today, where tens of thousands of different people around the world are using our CartoDB Editor and the CartoDB Platform (our set of APIs) to build amazing visualizations and geo applications.

And this Map Gallery is the celebration of it all, and the perfect showcase of what CartoDB is and what you can do with it. For the time being we have selected a few examples of things created with CartoDB; we’ll keep adding more things.

If you have created something with CartoDB and want to be featured in our Map Gallery just ping us (stories at cartodb.com) and let’s talk!

You can browse maps and visualizations by category, read about the visualization, and visit the live map. Not much more to talk about, but lots to see in Map Gallery ;)

Change the width of your infowindows

More love to our infowindows, and giving response to a very popular request: changing the width of our infowindows, directly in the CartoDB Editor. Up until now you could change the width of your infowindows with different combinations of not-so-obvious CSS and JS.

From now on, you can change the width of any of your infowindows in the Infowindow editor. You have a new “width” control, which lets you, mmm, change the width ;) We have predefined some min and max values so things are not easily broken, but if you have specific needs, you can just write any width you need (but be sure that the resulting infowindow fits in the canvas, and take into account mobile browsers!)

On a more geek side, note that for the width control to apply the desired effect, the div.cartodb-popup should also have the v2 class. The CartoDB Editor will apply this new class to previous and new infowindows, but if for some reason things are looking weird and your infowindow is not correctly resized, just double check for that class in the html code.

Recently: Infowindow on hover and icons for your markers

The NGO Aid Map—a new project powered by CartoDB

We are excited to announce the launch of the NGO Aid Map, an InterAction project developed by Simbiótica, a partner of CartoDB. ngoaidmap.org “aims to increase the amount of publicly available data on international development and humanitarian response by providing detailed project information through interactive maps and data visualizations”.

This interactive map and database shows the data generated by nearly 3,000 projects and more than 80 organizations around the globe, spanning sectors from health, agriculture, and economic development to education and even governance. The data comes from InterAction’s members, who manage billions of dollars’ worth of public and private funding every year.

How CartoDB helps nonprofits communicate and develop their missions

Julie Montgomery, Director of NGO Aid Map in her capacity as Director for Innovation and Learning at InterAction, describes in the project’s inaugural post, her organization’s historical use of online maps: “For the past six years, InterAction has been using online maps to help tell our members’ story. Today is important because we have launched a new global map on InterAction’s NGO Aid Map, one that will allow us to tell this story as it applies to all countries and all sectors

She enumerates some of the reasons why online mapping in general helps their mission, and why NGO Aid Map in particular matters to their work:

  1. NGO Aid Map is a tool for educating the world about the work of U.S. NGOs
  2. NGO Aid Map shows where the money goes
  3. NGO Aid Map improves decision-making
  4. NGO Aid Map gives context to project data
  5. NGO Aid Map encourages transparency

We find these outcomes laudable and inspiring, and have no doubt that harnessing the power of CartoDB would also help countless other nonprofits reach the same results as InterAction.

CartoDB gives NGOs one of the best available mapping tool and platform to develop these kinds of projects: DIY mapping lets anyone in an organization visualize and analyze data—with seamless importing, easy customizations, data segmentation, and public or private sharing—while our set of APIs allow developers to leverage the power of the CartoDB Platform to store data on the cloud without limits, integrate visualizations in any web app with easy-to-use and flexible APIs, and show data dynamically over maps.

Developed by Simbiótica, a CartoDB partner

NGO Aid Map has been developed by Simbiótica, a CartoDB Partner. The network of CartoDB Partners keeps growing, and they develop more and more interesting projects using CartoDB. Do you develop applications or create visualizations, and want to be a CartoDB Partner? Ping us!

Map of the Week: Areas prone to flooding in Norway

Welcome Øyvind Bye Skille to our Map of the Week series. Øyvind works for NRK (the Norwegian Broadcasting Corp.), the public broadcaster of Norway which produces content for both radio, TV and digital. Almost 9 out of 10 Norwegians use content from the NRK every week (88% in 2013). All development and production of web content these days require it to work both on mobile devices and desktop computers.

Several areas of central Norway were affected by flooding and most news media in Norway had their reporters out literally getting their feet wet. Inside the newsroom a question came up in an editorial meeting to the group of reporters Øyvind is part of: How can you contribute with content regarding the floods in central Norway?


We talked a bit about flood barriers, regulation of house building and where floods would hit the hardest. When discussing the last question about which areas were prone to flooding, and how we could tell our readers about it I remembered there might be some good datasets showing that as GIS data.

The idea was to visually show our web users something they could not get the same way in a news story aired on TV. From playing around earlier with other projects using maps and CartoDB I had good experiences with using datasets projected and styled on a map of Norway.

This time I made two sets of map visualizations:

1) How areas hit by flooding in central Norway this spring would get affected at different flood levels as predicted by Norwegian government (50, 100 and 200 years flooding)

2) How areas waiting for flood barriers would get hit by flooding as predicted by the government directorate planning for national flood contingency

The story of areas waiting for flood barriers were produced as a collaboration between a radio reporter and me as web reporter. She talked to sources who admitted over 50 cities and towns in Norway await flood barriers and other measures to minimize damages of floods. We also looked at a dataset showing existing flood barriers. The web stories with maps got promotion in nationwide broadcast on both radio and TV.

Data sources

In Norway the authority for water resources and national flood contingency are the Norwegian Water Resources and Energy Directorate (NVE). So I checked their web pages and found what I was looking for: Datasets showing floods happening statistically every 50, 100 and 200 years in Norway. The datasets can be found here: Geodata from NVE. English description of data from the authority here: NVE’s geographic databases

NVE call these maps of areas prone to flooding “Flood inundation maps”. They also share more than the flooding zones as open data, and most of the data have good descriptions (in Norwegian at least). At the download page on nve.no you find the flooding zones under «Flomsoner».

There you find one dataset for each flooding level from the one happening every 10 years to the massive 1000 year flood. The Norwegian Water Resources and Energy Directorate provides the data in two formats. SOSI, which is a special geodata format from Norway, and shapefile which is more common worldwide.

I choose the shapefile format as it is easier to use in tools like QGIS and CartoDB. Not too big shapefiles as a zip-package can be uploaded directly as a table in CartoDB. Then you typically get a table with geographic polygons and data.

In addition to the flood inundation maps provided as open data from Norwegian government NVE we used the the most detailed aerial imagery available in Norway.

Minimizing data in CartoDB

I started by downloading all the datasets for 50, 100 and 200 year flooding as shapefiles. The shapefiles come as zipped packages containing geodata, in this case polygons defining the areas calculated by NVE to be under water at the given flood level. They were all uploaded to CartoDB to their own table. These flooding datasets contain data for all of Norway. As a consequence they have a large file size. In order to not occupy too much of our CartoDB account I choose to filter them and only store the areas interesting during this flooding season.

My filtering was done by locating the flood zones visually in CartoDB, and then using another dataset with polygons for Norwegian municipalities to only include the wanted flooding polygons. In the CartoDB account we are using at the NRK we have some datasets of useful Norwegian entities, for instance the areas covered by Norwegian municipalities without ocean areas as prepared by my colleague Bjørn Sandvik – he has also shared it on Github.

Filtering with the two datasets was done with a SQL query similar to this:

The numbers at the end after the IN are Norwegian municipality IDs, and I chose the one I wanted to cover the necessary areas from looking at the map of Norway.

After running this SQL query I could save my minimized dataset by clicking «create table from query» in the CartoDB table view, and later delete the big table for entire Norway.

Understanding and selecting the right data

Now I had to review the dataset in more detail in order to be sure I used it correctly. Sources for this examination were the website of NVE and the dataset descriptions following the shapefiles as a pdf file. In these text you can see the polygons in the dataset not only defines the flooded areas, but also areas of rivers and areas that without it’s flood barriers would had been flooded.

In the dataset description one column is described like this:

Low points are those areas below the flooding level, but protected in some way the water will not reach it. So then I chose to select only the polygons defined as Area in danger of flooding (Flomutsatt areal = 1). Like this with a SQL query:

SELECT * FROM flomsoner WHERE symbol = 1

In addition to these checks of the data by reading the documentation I contacted the NVE directorate in order to get some advice and ask if the data were up to date. A person employed at NVE then could tell me a flood barrier installed in 2007 in one of the areas I were examining had not been included in the dataset. He provided me with a PDF with the new flood barrier and how it affected the areas flooded at a given level.

With this information I could manipulate some polygons to make the older dataset as close as possible to the pdf. This can be done directly in CartoDB by clicking the polygon covering the area and choosing “edit data”.

I did achieve an area not flooded on the inside of the barrier, but some small areas elsewhere in another area lost their definition as flooded. Though there were few buildings in those areas and I found it as a compromise.

Styling the maps and background layer

All the flood inundation maps were styled in CartoDB before being imported to our own custom map API. In CartoDB I used two different styling approaches:

  • Maps with only one flood level styled with a simple polygon fill and outline
  • Maps with three flood levels styled with several polygon fills

The maps showing where The Norwegian Water Resources and Energy Directorate fear a flood and wants more flood barriers were made with one flood level of 50 year flooding. This was chosen to make them simple, and also not to exaggerate and be accused of promoting fear.

To show the inundation areas the polygons was defined with polygon fill and outline in blue representing water. The fill was added with low opacity in order to let the users see the background aerial imagery in the layer beneath.

We used CartoCSS like this:

NRK has extended the functionality of CartoDB by not embedding the visualizations directly, but through CartoDB.js. We have created our own custom map API based on Leaflet, allowing us to mix and match data from various providers. For our flood maps, we’re using the most detailed aerial imagery available in Norway.

Our mapping API also allows us to improve the user experience by only showing a static map image, generated by PhantomJS, on small screens or mobile devices. The mapping library will only be downloaded if the user interacts with the map. To avoid a “scroll trap” in our articles, we’re only allowing panning and mouse wheel zoom when the map is in fullscreen mode. The extendability and open philosophy of CartoDB was important when we decided to use it in our mapping stack.

The maps of different flood levels in the already affected areas this spring we used the same approach, but put three layers on top of each other in the CartoDB visualization view. The layers were styled with different colour variations of blue, and we added a legend to explain the different flood levels.

This approach kept the watery feeling of the visualization, but user testing showed some difficulty for readers keeping the flood levels apart. For further mapping we plan implementing an interactive layer switcher in our in-house mapping stack. With such a feature the users could have shown only one flooding level at the time and checked the difference themselves by clicking the layer switcher.

For Norwegian readers most of this article is also available at nrkbeta.no

More detailed geospatial analysis with CartoDB

Earlier reporting done by the NRK using CartoDB include extensive geospatial queries and analysis.

In the beginning of May we published a story presenting the number of people living outside 30 minutes flight time from ambulance helicopters. The number was calculated using CartoDB, facts about Norwegian ambulance helicopters and population data on a grid provided by Statistics Norway: 166.000 in Norway live outside 30 minutes flight time.

We also published an interactive map where web users could check the flight time to anywhere in Norway by clicking the map or providing the nearest postal code. Part of the backend for this interactive map was also CartoDB.

More about these other maps and how we made them: Ambulance helicopters, methods and analysis (Norwegian) (Google translate)

Web stories with flood maps published on nrk.no:

Several cities and towns waiting for barriers fearing new floods

See how cities in central Norway would get affected by a major flood

Mjøndalen with 50 years flood

Hamar with 50, 100 and 200 years flood


Check out our other Maps of the Week, and jump right into CartoDB to start creating amazing maps like this.

Stay tuned and subscribe to our mailing list

*We promise we'll never share your email address with anyone