A choice of useful links to every (wannabe) journalist programmer

Jack Lail is News Director of Innovation for the Knoxville News Sentinel. Amongst other random mumblings, he wrote about Some data on database journalism. Because he was speaking to a journalism class about Computer-Assisted Reporting, Computational Journalism and Open Government Data Advocacy efforts and needed some material.

Foto credit: www.sxc.hu. Modified.

Go there an read the full post. Honoring his 3.0 CC license, I republish below a choice of his links, useful to every (wannabee) journalist programmer.

Computer-Assisted Reporting:

Tools

The new era of digital news publication (answers the question: what is a journalist programmer?)

In the new era of digital news publication does still apply the journalism jargon developed along two centuries of printing press?

In a word: no.

In an article published a week ago at ClickZ, Heidi Cohen synthesizes the 3 Cs of digital news consumption:

What makes a platform relatively desirable to a news consumer? Here are the three Cs of what they want from news:

  • Customized. It’s tailored based on individual needs, interests, location, political views, and other factors.
  • Curated. It’s selected by a combination of professional news editors and one’s social graph. This serves as a lens for which information is viewed and from what perspective.
  • Contributory. It’s enhanced and modified by the addition of opinions and sharing of information through various forms of social media (e.g., Facebook, Twitter, blogs, and comments).

Jeff Jarvis wrote that a while ago (in 2008), but almost anybody in the newsrooms was reading: The building block of journalism is no longer the article.

Divide the journalistic product, whatever it will be, no longer in articles, “breaking news”, respectful “opinions” from well established opinion-makers. Digital news publication are divided in three main areas, or keywords if you prefer. They are Firehose, Trend and Memory.

  • Firehose
    All the headlines, real time. Breaking news fit here, tips fit here, news fit here, new products, new events, new everything. Think Twitter, Facebook and news aggregators. You must publish first for the firehose. Your firehose (newspaper/blog feed, Twitter timeline) as well as the firehose defined by the consumer (customization, on Cohen’s description).
  • Trend
    What are people talking about, what is interesting? Generic or niche, what are *my* people discussing, being impressed by, affected by? The headlines or events “chewed” and considered, be it by conversations or explained in posts all over the web. Curating happens here. So does contributory journalism (a.k.a. citizen journalism).
  • Memory
    No understanding and education exist without memory. The memory is the data. The headlines that were trends in the past. Whenever it deserves a better explanation, the occurrence (what made the news) will be compared with the memory bank.

A journalist programmer is a journalist trained with informatics tools and skilled in detecting, editing and publishing trends, and knows which memory databases to query, and how to query them, every time she/he is called to explain or help understand what happened.

Harry Potter’s Daily Prophet is not good enough: present you Text 2.0

Speaking about all that media frenzy with iPad, how can reading evolve on digital devices? Do you think (as I do) in video instead of pictures like in The Daily Prophet read by Harry Potter?

OK, but think again. Think in a “page” where “sounds, images and other types of ambient events can be triggered appropriately according to the currently read position“, according to the movement of your eyes, according to the questions you ask the “book” or “newspaper” you’re reading.

Think Text 2.0. As they put it: “Text 2.0 is interactive, but without the need of explicit control. By a mere reading detection the system recognizes the current point of attention and enables to create an atmosphere full of sentiments, for example by using sounds of wind and water, emotional music or background themes“.

Using market-available eye-trackers along with HTML, CSS and JavaScript, they created a reading enhancement technology called Text 2.0.

From h+ mag’s Does This Headline Know You’re Reading It? (which was slashdotted):
This is not simply a case of using infrared light, a camera, and eye movement to move a cursor and click buttons: Text 2.0 infers user intentions and enhances the reading experience in far more complex ways. Reading certain words, phrases, or names can trigger the appearance of footnotes, translations, definitions, biographies, even sound effects or animations. Ask how a word is pronounced and you get a verbal answer. If you begin skimming the text, it fades out the less important words. If you glance away, a bookmark automatically appears, pointing to where you stopped reading

See the video below, you’ll understand. This guys are sustained by the German Research Center for Artificial Intelligence.

Andrew Keen use to say a lot of things

Is Innovation Fair? Andrew Keen Says No – titles Dana Oshiro a pertinent article on ReadWriteWeb.

Andrew Keen use to say a lot of things. I usually disagree with Andrew Keen. But this is also true: I *always* read Andrew Keen. Because Andre Keen makes me think.

You can’t get nuggets of truth in 30 seconds on Twitter…Skepticism requires deep thinking. We have an increasing nihilism when it comes to traditional authority and yet few of the new authorities are doing the reading or groundwork. …When we simply assume that all traditional structures are wrong, we risk the populism of a Sarah Palin…

Well, maybe Keen & Others looked for “truth” in Twitter, but not everybody did. I didn’t. And I’m not sure skepticism requires deep thinking. I can be skeptical by reaction, and reactions may occur (they generally do!) in a fraction of time. In a tweet.

Nevertheless, Keen made a point. And this is worth reading:

“Innovation doesn’t lead to justice and fairness. I’d argue there is a more dramatic inequality now then there ever was during the industrial revolution. We have fetishized change, but we are unfettered. If anything, the new media is less transparent and less accountable…I don’t have a problem with Twitter or new media, my problem is that digital utopians have dressed up their ideology to sound like democracy…Google has become the master of seeming like an altruistic and public company and yet laughing all the way to the bank.”

So, go there and read it.

Social media infographics (slideshow)

Oscar Berg loves “great infographics and so do I. He collected several social media-related infographics and come up with this nice slideshow, just look:

2010: The year open data went worldwide

2010 will be remembered as the year open data went worldwide. Sir Tim Berners-Lee believe it and so do I. In this short TED talk Berners-Lee talks about Haiti, yes, but also about data effect things.

Newspapers and online media *can* lead the process of putting all that public data into nice visualization schemes and useful interactive mashups — if they hurry. Or the people — bloggers, social media gurus, corporations, governments and non-profit organizations themselves — will do it, replacing that historical journalistic roll of gathering, filtering and presenting, you know, news.

Feed and newsletter readers, this link to watch the video)

An excerpt:

Does this data effect things? Well actually let’s get back to 2008. Look at Zanesville, Ohio. Here is a map a lawyer made, put on at the water plant, seeing which houses are there, which houses have been connected to the water? And he got, from other data sources, information to show which houses are occupied by white people. Well, there was too much of a correlation, he felt, between which houses were occupied by white people and which houses had water, and the judge was not impressed either. The judge was not impressed to the tune of 10.9 million dollars. That’s the power of taking one piece of data, another piece of data, putting it together, and showing the result

Raw Data Now

Don’t lose the first TED Talk by Tim Berners-Lee about data. It was a year ago, data.gov was nothing but an intention and data.gov.uk wasn’t even a dream.

See it on TED, or here:


Feed and newsletter readers, this link to watch the video)

Future Media (is here. Now)

“If media is in the infrastructure business, it is in the wrong business.”

“The only wat to earn money in online publishing is by having a strong and identifiable brand with a unique offering“.

“Start seeing your online publication as value provider, not content provider“.

“This is an opportunity we have been waiting for, for fifteen years. We have all these visual tools at our disposal, to tell all these stories, in a way that is efficient, that is multi-dimensional“.

Journalists, media guys, be warned: lose at your own, severe risk this presentation by Helge Tennø (via Armando Alves).

The essential *nix commands

I discovered The 15 Essential UNIX commands written almost 5 years ago by Pete Freitag.

In 2005 talking to journalists in *nix commands was a bit of a daunting. But nowadays they are more known: much more journalists and media guys run Linux servers or Macs with OS X.

First read Pete’s choice, then mine:

1. man – show manual for a command, example: man ls hit q to exit the man page.
2. cd – change directory, example: cd /etc/
3. ls – list directory, similar to dir on windows. example: ls /etc, use ls -l /etc to see more detail
4. cp – copy a file or directory, example: cp source dest if you want to copy a directory use the -R option for recursive: cp -R /source /dest
5. mv – move a file, example: mv source dest
6. rm – remove a file, example: rm somefile to remove a directory you may need the -R option, you can also use the -f option which tells it not to confirm each file: rm -Rf /dir
7. cat – concatenate, or output a file cat /var/log/messages
8. more – outputs one page of a file and pauses. example: more /var/log/messages press q to exit before getting to the bottom. You can also pipe to more | more from other commands, for example ls -l /etc | more
9. scp – secure copy, copies a file over SSH to another server. example: scp /local/file user@host.com:/path/to/save/file
10. tar – tape archiver, tar takes a bunch of files, and munges them into one .tar file, the files are often compressed with the gzip algorithm, and use the .tar.gz extension. to create a tar tar -cf archive.tar /directory, then to extract the archive to the current directory run tar -xf archive.tar to use gzip, just add a z to the options, to create a tar.gz: tar -czf archive.tar.gz /dir to extract it tar -xzf archive.tar.gz
11. grep – pattern matcher, grep takes a regular expression, or to match a simple string you can use fast grep, fgrep failure /var/log/messages, I’m usually just looking for a simple pattern so I tend to use fgrep more than regular grep.
12. find – lists files and directories recursively on a single line, I usually pipe grep into the mix when I use find, eg: find / | fgrep log
13. tail – prints the last few lines of a file, this is handy for checking log files tail /var/log/messages if you need see more lines, use the -n option, tail -n 50 /var/log/messages you can also use the -f option, which will continuously show you the end of the file as things are added to it (very handy for watching logs) tail -f /var/log/messages
14. head – same as tail, but shows the first few lines the file
15. vi – text editor, there are several text editors such as emacs, and nano, but vi is usually installed on any server so its a good one to learn. To edit a file type vi file to edit a line press Esc i then to save changes and exit use Esc wq, or to quit without saving use Esc q!. There are a million other commands, but that will enable you to edit files at a basic level.

I’m a vi guy

I use cd, ls (with some customization via .bashrc), cp, rm, scp, grep, tail — and of course vi. I’m a vi guy. Even on my MacBook, where I usually write in a registered TextMate text and code editor, sometimes it is easier to edit with vi. I use it in a very simple way, I don’t know most of its commands and functions (nobody does).

My adds:
w – displays information about the users currently on the machine, and their processes. Useful.
top – provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel. When you need to debug a server this command is essential.
ps – report a snapshot of the current processes. I use it with ax and some pipelines to quickly get precise info (like: ps ax | grep httpd | wc -l to see how many Apache processes are runing).
crontab – cron is a time-based job scheduler and crontab is the configuration file that specifies shell commands to run periodically on a given schedule. I run a bunch of scripts (PHP and bash, mostly) and crontab is *the* way to organize the scheduling and other aspects, like logging and mail-logging their outputs. Repetitive tasks, like pulling a RSS feed and do things with it, are performed by scripts orchestrated within crontab.

Facebook: 6 million pageviews. Per minute.

Facebook’s 30.000 servers (and growing) serve 6 million pageviews per minute. Yes, 6.000.000 in each minute, or 100.000 per second, if you prefer.

For a data maniac and data visualization consumer like myself, this simple video from Jesse Thomas is a delight. Just watch (feed and newsletter readers, you may have to click here) The State Of The Internet:

Journalist programmer: the perfect mutant? No.

Journaliste-programmeur, le mutant parfait? asks Sabine Blanc at Owni.fr. Well, the short answer is no, but Sabine Blanc’s question is a figure of speech.

She is reacting to Ryan Tate’s important Hack to Hacker: Rise of the Journalist-Programmer. Both articles impose the journalist programmer as a trend. And this trend is the news — not the humoristic aspects of the trend.

The “simple” journalists don’t need to tremble: there is no super-powered mutant professional coming from a high-tech spaceship to get their jobs ;)

I partially agree with the conclusion: “Si le double profil ne devrait donc pas devenir incontournable, il est aussi évident qu’une bonne louche de culture web supplémentaire ne ferait pas de mal à la profession, dès l’école ou en formation continue. Et pour ceux qui veulent aller plus loin, une spécialisation programmation.

No, the double profile is not inescapable. Far from it. But I don’t expect that programming becomes a matter of general culture. It will be just a skill, maybe a specialization, even if I don’t like the expression. Is photography a specialization of journalism?

Maybe the history can provide some clues.

On the early nineties a similar change occurred when the infographics were introduced. At first, only a very small group of illuminati in the newsrooms understand the appeal (for the eyes) and the potential (for showing in a glance big chunks of information) of the infographics. It passed some time until most journalists accept the infographics in their articles. An even more time until every print journalist wish an infographic illustrating his/her story.

The infographics also born from information technology: they proliferate with Desktop Publishing — you know articles and news were typed in typewriters before, right?

Did everybody in the newsroom become an infographic?

Of course not. But the process made its way and now every newspaper was an infographics section and buys infographics from news agencies. And, glad you asked, yes, back those days a bunch of journalists was terrified by the prospective of being forced to learn CorelDRAW or Illustrator, and draw!

Who are the infographics ? Are they converted journalists? Designers? Artists?

You don’t have a simple answer for that. Some of them have a journalistic background, others came from art schools. In some newsrooms they are marginalized. On others they are powerful. But the infographic become a journalistic process. A pretty nice one, be it on static on paper or animated on the screen.

Expect the same pattern here. Some old journalists (like myself) will learn code, but we are a very restricted club. Some new journos know basic PHP they learned with blogs and WordPress; some of them will become fine journalist programmers. Some programmers will fall in love with journalism — or they just find a job in new media.

The journalist programmer is just another branch. The next branch. Code is just another tool to tell a story.