Did you know that CartoDB runs atop PostgreSQL, the most advanced database in the world? Which also happens to be open source? And the fastest growing database in the market?

We actually contribute extensively to our favorite extension of it, PostGIS.

A lot of what is possible in CartoDB is thanks to this flexibility and extensibility of PostgreSQL. (And it still amazes us, all the cool things that can be done in just good, old, plain SQL (size sorting, distance queries, or digging into data).

PostgreSQL has some of the most advanced and replicated technologies, with extremely powerful extensions and analysis SQL. But even more importantly for us at CartoDB, with PostGIS, PostgreSQL offers exceptional geospatial support. PostGIS performs better and with superior functionalities than Oracle, MongoDB, SQL Server, and well, any other database out there. And the fact that PostGIS is just an extension to PostgreSQL demonstrates how extensible and powerful that database is.

Every year the PostgreSQL community gathers together to discuss the future of our favorite database and share ideas. This event is called PGCon, and it happens in Ottawa this week! Our CEO Javier de la Torre will be there to hear from others and share some of CartoDB’s experiences.

Or,  if you ever want to talk about maps and PostGIS in general please ping us. We’d love to show you CartoDB, talk about maps with millions of records, hosted PostgreSQL services, etc.

But hopefully — see you in Ottawa!

The open source geospatial community has some of the best people and the most exciting ideas. That’s why we are always excited to take part in one of a few conferences that bring so many of those great people and amazing ideas into the same place for a few days. This Wednesday marks the start of FOSS4G-NA, a fairly new meeting meant to give the North American FOSS4G community an additional and slightly easier way to get together outside of the big FOSS4G conference. It has been a great addition.

Right now, we are putting the finishing touches on a couple of presentations we will be giving at the conference. The first presentation is a lightning talk titled, The future of geospatial data formats at CartoDB. In that talk, we want to show you some of the ways we are using novel formats and the modern web to push geospatial data visualization. The second is titled, Building Geospatial Applications that can read and write data to CartoDB without proxy. In that presentation, we are showing how CartoDB can be used without any additional backend infrastructure to do some interesting things on the web.

If you can’t make it, be sure to follow the conversation on Twitter. If you can make it, find us (@CartoDB or @andrewxhill) and let us know what you are up to using CartoDB or ask us any questions you are dying to get answered. Looking forward to seeing everyone there!

While many CartoDB users arrive to the service with data on hand others look to use CartoDB to host and map data from ongoing collection. For those users, we offer a number of useful client libraries and tutorials for using our APIs. For businesses, scientists, and students that still want a little easier way to collect data, we thought we would put together this tutorial covering how to collect data with Google Forms and have it inserted directly into a CartoDB table.

In our example, we create a simple form to collect people’s favorite color and their location in the world. Using Google Forms plus a small Google App Script, we then insert those results into our CartoDB table where we have a live map that shows the latest and greatest results to the world. Be sure to fill the form out yourself and see your vote show up on the map!

We have included all the code plus the screencast for you to create your own forms for data collection. It only takes about 5 minutes! Have fun and remember to share the cool things you come up with!

We have been discussing a lot lately how we can summarize CartoDB in a short and sweet sentence. A lot of adjectives have been tested. One of the options we have liked the most is, CartoDB allows you to render your data on a map. As simple as the phrase sounds, it packs a lot of meaning. Let us explain…

How CartoDB serves geospatial data, the common

At its core, CartoDB gets the data from a database and renders it to tiles. Tiles are a clever solution where images of data instead of complete datasets are transferred over the internet, saving a lot of bandwidth. They also are created on a regular grid that makes them perfect for caching. Once delivered to the browser, those tiles are rendered by any client side API of your liking (e.g. Leaflet, Google Maps, OpenLayers, ModestMaps). Besides being a clever solution to displaying large-scale data on the web, rendering tiles is fast and compatible with old browsers and mobile devices.

Tiles are not a magic bullet for maps on the web. Most obviously, when we don’t have a lot of data there isn’t much gain from tiles versus sending the real data itself. For example, if you need to render 100 points, you will need 100*4*2 (4 bytes per float, latitude and longitude), 800 bytes. It is very likely that the tile or tiles you would need to render these points would take much more than 800 bytes. You also get all the benefits of dynamically changing your data (e.g. adding, removing, or moving points) and you can add things like hover effects that can not be done with tiles alone.

Luckily, CartoDB is way more than just tiles.

How CartoDB serves geospatial data, the non-tile way

CartoDB has a nice API to access to the data, the SQL API. It takes a SQL statement and returns the resulting data. You could use it for simple things like, get the points in your table, to complex things like we do with Torque. Torque is built around the idea that a powerful SQL function can be run to turn any CartoDB table with temporal geometry into a moving visualization in the browser. That can’t be done with tiles.

The SQL API also gives developers a few options for data formats, those include a flat JSON format, TopoJSON and GeoJSON. The GeoJSON format is meant for us mappers, it allows you to transfer geographic information (including metadata) using JSON as container, but in a predictable schema that many of your client side mapping libraries already know. Sounds good, eh?

Getting the data from your cartodb account to your JavaScript application can be done really easily using CartoDB.js. Here is an example of the JavaScript API:

Rendering vector data using Leaflet

Using Leaflet as an example, we render that GeoJSON data on our map with just two more lines of code:

You can see it running here.

Rendering vector data using Google Maps

Google Maps does not support GeoJSON natively but there is a library that fills the void, GeoJSON-to-Google-Maps. The library works in a different way than Leaflet and takes a few more lines of code. You can see a simple example with source code here.

CartoDB is built for your dynamic data

Okay, that was fun but the best thing is yet to come. CartoDB is a dynamic service and that means you have some powerful flexibility in how you retrieve your data for the web. You could execute queries to only get a subset of your data. Or you could query for only the latests data inserted into your tables. You could optimize your maps by changing the quality of the data depending the device your viewers are using. Or you could animate your data if it is linestrings or points with timestamps. This is where the SQL API really gets powerful.

Filtering the data

Imagine you want to get the countries of a particular size, say 1,000,000 Sq Km. Here, we’ll use the SQL API with a little bit of geospatial filtering (ST_Area is going to return meters, so we divide by 1 Sq Km):

Here, we can query for only the points close to known location, specifically the wifi hotspots nearest a viewer.

Even though our blog gives a lot of attention to maps, CartoDB is a great tool for a lot more than just maps. We have seen in the past how the CartoDB APIs can do all sorts of dynamic queries to CartoDB hosted data. While dynamic queries CartoDB can be geospatial in nature, even returning GeoJSON formatted results, we haven’t spent much time highlighting the fact that they don’t have to be geospatial. That is why we are excited by the latest project released by the Climate Policy Initiative, the Policy Climate Interactive.

The Policy Climate Interactive is an online tool, focused on the key economies driving both climate change and climate policy in the world. The report highlights both the good and the bad through the use of interactive data visualization driven by the CartoDB APIs. No other platform available gives the flexibility to dynamically query hosted data in a way that you can create client side visualizations so simply.

In this case, the D3 framework was used to build some really beautiful graphs of the data underlying global climate policy. We were really impressed to see how beautifully CartoDB works with D3 for doing these data visualizations without any maps! This project highlights how powerful CartoDB can be even for non-map based data visualization. Take a look through some of the results,

The way the project uses the D3 library and data coming directly from CartoDB is cool for many reasons. A clear one is the data curation and management flexibility it give the project administrators. Administrators can easily add and modify the source data on CartoDB and the results will immediately appear in the graphs online! CartoDB also gives users the ability to create download links for particular subsets of data, by adding a link to the API query with and indicating the export format (e.g. CSV, Excel, etc).

We are excited to share this project and are honored to have CartoDB be part of such an important project. Congratulations to [Climate Policy Initiative](http://climatepolicyinitiative.org/) for delivering such a forward thinking website and data publishing platform.

For years we at CartoDB have been dedicated to visualizing geospecific data to gain analytic insight and tell compelling stories. Our real-time rendering and time-lapse technologies have helped illustrate phenomena all over the earth from Barcelona traffic to Kenyan elephant movements to the history of meteorite strikes on the planet. Now for the first time CartoDB’s power of dynamic mapping is also enhancing a virtual world with massively multiplayer online game eRepublik.

eRepublik is a popular free-to-play strategy game with hundreds of thousands of users globally. Players in eRepublik’s New World claim allegiance to a specific country, for which they build companies, develop diplomatic strategies, and fight on battlefields for more territory. While maps are an especially apt visualization for this kind of gaming application, eRepublik had a hard time to provide them for so many players with any useful accuracy. The game’s very popularity and the volume of rapidly changing content posed the greatest obstacle to successful mapping. CartoDB, however, gives eRepublik a means of producing accurate maps for their many users for a fraction of the cost charged by other SaaS mapping companies.

CartoDB, unlike other mapping platforms, is in fact designed to handle dynamic content at large scales efficiently. With our API eRepublik no longer needs to generate new maps after each player’s action. Instead, as the game progresses eRepublik simply modifies the map data, and the platform renders the changes automatically. Map content gets updated at least once every twenty minutes and players are able to see a visual representation of the game’s current state with practically real-time accuracy. CartoDB handles approximately two million map views in eRepublik daily.

We’re pleased at CartoDB to share this use of our platform by eRepublik both because we’re proud of how it showcases our exceptional dynamic rendering and scaling capabilities, and because for us it’s new territory. Games like eRepublik demonstrate that not every community organizes around real space, and geospatial data is not the only kind that produces interesting map stories.

Let us know if you have more ideas for alternative-space uses of CartoDB, and as always, don’t be shy about sharing your own map stories!

Today we here from Chong Zi Xin and a team of hackers that took part in the recent, Urban Data Challenge. The team reached out a few months ago about using CartoDB for their challenge entry and the other day we found that their entry had won 3rd place. We were really impressed with some of the maps they produced, including perspective views with integrated bar charts (wow!). 

The Urban Data Challenge is a competition that aims to improve transportation through the visualization of urban data sets by drawing meaningful insights. Public transport data such as trams, buses, bicycles and pedestrians from Zurich, Geneva and San Francisco is provided for participants to merge and compare mobility data sets from these three cities.

The project, titled “A City’s Heartbeat”, done by core members of three start-ups, won third prize in the Urban Data Challenge. The project utilizes Geneva’s trams and their passenger flow information to help improve urban mobility, tourism experience as well as businesses (such as cafes near tram stops) to make better decisions.

Our first challenge, is to render big, time-series data of tram movements over a two day period. This temporal mapping was achieved by using the CartoDB SQL API and reusing code from their Torque examples. The basic idea is to treat the database table as a cube, while only downloading and visualizing a few slices of the data at a time. It worked beautifully. 

This year’s BigApps competition is kicking off right now in New York City. The competition challenges developers and designers to create applications that help the city tackle big issues, help make people’s lives a little better, and help people engage in the city around them.

We have been supporting the competition for a few years now, and we have seen cool applications like Scene Near Me developed using CartoDB. For this years challenge, the City has opened some great new datasets and has even expanded the rules of what you can use to participate on the challenge. As part of that expansion, CartoDB is now one of the official APIs and developing your app using CartoDB qualifies you to enter into the competition. We would love to see one of you win any of the great prices.

We are at the NYC BigApps 2013 Expo and Hackathon Weekend at eBay presenting our API and helping developers with anything geo and maps. If you are around, Andrew Hill can give your team a free upgrade to a Magellan account on CartoDB. 

We can imagine lots of different applications that can be developed for the great city of New York.  There are some areas in particular where we think CartoDB can really help participants. If your application has:

• Maps that are interactive and that are dynamic
• You need to find what is the closest subway, park, museum or whatever to a location.
• You what to intersect location data from phones with any other dataset, in real-time.
• You need a backend to store your data and be able to query on it live.
• You need to analyze multiple datasets, merging them via location. For example what neighborhoods are more “bike friendly”.

We are really excited on this challenge and we look forward to help all participants with their GEO needs! Good luck everybody.

As some of you may already know, Newsweek / The Daily Beast has been using CartoDB for some time now, and as such today’s blog post comes from Michael Keller of Newsbeast labs. We’d also like to take the opportunity thank Michael for his amazing contributions to the CartoDB community. Thanks!  

A number of recent stories at the Daily Beast have had some kind of mapping component. We use them often to let people see how a national topic affects readers’ local areas.

I have been reusing code from former projects and so it was about time I standardized them into reusable templates with Leaflet.js. I released them on Github this week. 

I made three categories: basic map with hover states, hover states + hover infowindow, and all of that with templated infowindows using Underscore.js.

In each of these categories you’ll see a template for a point map, a polygon map, and a map with both points and polygons.

Some features:

• On point + polygon templates, the polygon hover state turns off when you hover over a point.

• Hover windows follow the mouse and respect the boundaries of the map-canvas. I find it most useful to have hover windows close to the mouse so your eye doesn’t have to leave that map region to see that region’s details

• Templates with Underscore.js hover windows include sample formatHelper functions to act as a formatting layer between your data values and how you want them to display. For instance, you could store all your feature attributes as boolean variables and run them through various formatHelpers functions to return nice display strings.

• The hover states work by storing a simplified GeoJSON representation of that feature as a feature attribute. On featureOver, that GeoJSON is plotted as a vector using Leaflet.js.

• Point + polygon templates add a secondary style class to hover windows when hovering over points to differentiate from polygons.

If you have any questions, I’m at @mhkeller. If you have improvements, pull requests at http://github.com/mhkeller/cartodb-templates

Over the past months you have probably come to realize, we love data that moves. Moving data can make your maps and visualizations really come to life and data you can use to build these visualizations can come in a lot of forms. Whether it’s data that crawls and zips across a map through time, data that bursts and comes to life as you watch, or data that changes form or intensity, it all can lead not to some really amazing visualizations.

Seeing data come to life on your computer screen is more than just fireworks that people find pretty. Showing data move on maps isn’t new, but we have really been trying to push new research into how to effectively display dynamic data on interactive maps. Seeing and also interacting with data that changes through time can be incredibly useful for gaining insights into your data and communicating those insights to a broader audience. The application of these visualizations can be anywhere from business intelligence, to smart grid development, to conservation planning.

That is why we were particularly excited to see a recent visualization from Jeffery Stephens and Space for Giants. Jeffery used Torque and CartoDB to start looking at how elephants moved across the landscape. In particular, these elephants have a bad habit of crossing a fence from protected areas into nearby farming areas. It is a bad habit that can lead to really bad consequences and fixing it requires understanding how and where it is happening and organizing a multifaceted solution. To help share the issue with colleagues, collaborators, and others, they put together a really unique visualization of the elephants moving across the landscape.

In the visualization above (click the image to see the live map), a year’s worth of elephant movement was reduced into a single day. Next, the data was mapped over a 24-hour clock, showing the dominant patterns of where and when the elephants were crossing the fence. We absolutely love this map, and hope it can help the wonderful work of Space for Giants. We weren’t alone, there was a really nice write-up in O’Reilly the other day too.If you want to try your hand at similar visualizations, take a look at our Torque library available on GitHub. We are still working hard to incorporate this directly into the CartoDB user-interface, but in the meantime, you can fairly quickly get a look at your dynamic data using the sandbox tool. We would love to see any interesting visualizations you come up with!