Scraping for everyone

Scraping for everyone

by Sophie Rotgeri, Moritz Zajonz and Elena Erdmann

One of the most important skills for data journalists is scraping. It allows us to download any data that is openly available online as part of a website, even when it’s not supposed to be downloaded: may it be information about the members of parliament or – as in our christmas-themed example – a list of christmas markets in Germany.

Extracting geodata from OpenStreetMap with Osmfilter

Extracting geodata from OpenStreetMap with Osmfilter

A guest post by Hans Hack

When working on map related projects, I often need specific geographical data from OpenStreetMap (OSM) from a certain area. For a recent project of mine, I needed all the roads in Germany in a useful format so I can work with them in a GIS program. So how do I do I get the data to work with? With a useful little program called Osmfilter.

From the data to the story: A typical ddj workflow in R

From the data to the story: A typical ddj workflow in R

R is getting more and more popular among Data Journalists worldwide, as Timo Grossenbacher from SRF Data pointed out recently in a talk at useR!2017 conference in Brussels. Working as a data trainee at Berliner Morgenpost’s Interactive Team, I can confirm that R indeed played an important role in many of our lately published projects, for example when we identified the strongholds of german parties. While we also use the software for more complex statistics from time to time, something that R helps us with on a near-daily basis is the act of cleaning, joining and superficially analyzing data. Sometimes it’s just to briefly check if there is a story hiding in the data. But sometimes, the steps you will learn in this tutorial are just the first part of a bigger, deeper data analysis.

JavaScript: Loop through a Google Spreadsheet

JavaScript: Loop through a Google Spreadsheet

In our post Coding marker maps with Leaflet.js we filled a map visualization with information given directly in the code, but we also learned how to loop through data in a JSON file to create markers and pop-ups. This time, we will loop through an public Google Spreadsheet. We will do this with the same example map as in the previous post. But this little trick will not only help you with building marker maps but can come in handy for any JavaScript application you need to automate the handling of your data.

So let’s start right away with setting up a data table with Google Spreadsheets.

How to encrypt your emails with PGP keys

by Moritz Zajonz 1 Comment
How to encrypt your emails with PGP keys

Important Notice: Considering the recent disclosure of vulnerabilities in popular e-mail clients like Mozilla Thunderbird, we decided to delete this post. The current PGP implementation in email clients has vulnerabilities, that haven’t been fixed for now and will take time to get fixed. For more information about the technical side visit efail.de and for a detailed explanation, read the post by the Electronic Frontier Foundation. Thanks for your interest in this topic! We will update this post when new info is available.

 {Credit for the awesome featured image goes to Phil Ninh}

JavaScript: Coding marker maps with Leaflet.js

JavaScript: Coding marker maps with Leaflet.js

Showing locations on a map can be pretty cool to provide some context for your story or to give your reader an overview of where the story takes place. A good way to build a simple, yet responsive and professional looking map is to use the JavaScript library Leaflet.

In an earlier post, “Your first choropleth map“, we used Leaflet as well, but coded the map using the Leaflet R package, which works like a wrapper to translate the more common Leaflet functions into R syntax. It’s very useful if you’re more used to R syntax and don’t want to learn JavaScript anytime soon. But using the original JS library and coding the map with JavaScript will give us way more freedom when customizing the map, which is why we’ll try it that way today.

As an example, let’s start with a map of the locations of some data journalism newsrooms in the German speaking area. As always you can find all the code of this tutorial on our GitHub page.

This is what the finished map will look like:

HTML, CSS & a little JavaScript: The Basics (Part II)

HTML, CSS & a little JavaScript: The Basics (Part II)

Part I

This is part two of our tutorial on HTML, CSS and a little bit of JavaScript. In the last part, we learned about the basic functions of those three languages and have gotten to know a few useful HTML commands. If you’ve already read part one or you know all of that stuff anyway, this is the perfect spot for you to continue – by learning about CSS and how to implement JavaScript libraries into your webpage. Let’s get right to it!