A Sleepless Night From Hell

That was easier than I thought. But as I got lightning running I noticed other scripts failing.

I’m working today. There will be afternoon/evening thunderstorms in Nebraska. My tools are ready after what was a mainly sleepless night from hell.

My former North Branford middle school weather watcher, now meteorology professor and noted authority on tropical weather Dr. Bob Hart, helped me over a mapping problem.

I couldn’t figure out how to plot lightning. Months of futility.

Lightning is different from just about every other map I make. Most are contours, areas that can be surrounded by drawing a line. Lightning is plotted as points, often colored or sized to show frequency.

The lightning data I get divides the world into 1km squares (about 6/10 mile on a side) then calculates how many strikes occur in each. I fetch 24.5 million squares of data every two minutes — the vast majority empty (no lightning).

On a national or even a regional map a 1km “point” is easy to miss. I needed something bigger. Bob figured out how to plot the points as characters. I’m going to add his explanation. He has major league coding chops. What looks like a single line of code is really five separate instructions not all of which I understand.

extracts in csv format all the values of flash count that are non-zero, replaces the commas by a space, and then sorts by the flash count so that higher flash counts are drawn on top of lower ones in the grads script.   this is then redirected to a text file out.txt

wgrib2 MRMS_NLDN_CG_030min.latest.grib2 -undefine_val -9e20:0 -csv - | cut -d',' -f5-7 | egrep ',' | sed 's/,/ /g' | sort -k 3 > out.txt
Demo lightning map from Bob’s generic script

My TV version of a Texas lightning map

My job was to take Bob’s generic code and modify it to run in my environment. That was easier than I thought. But as I got lightning running I noticed other scripts failing.

Somehow I deleted the entire directory containing my basemaps. There was a backup. It was missing a few recent maps. It took a few hours to remake them.

Also deleted were a bunch of my most important scripts. These are programs that call other programs. Backups for this weren’t as easy to come by.

I am such a slob I was actually able to scrape up most of these from discarded versions I never deleted. Most were abandoned when the Weather Service changed from http: to https:, so updating was simple. A good programmer would have deleted those! A good programmer would have better backup.

Most everything was running by 3:30 am. More problems would be discovered through the morning and quickly fixed. It was time to make the lightning maps TV pretty.

As a meteorologist I don’t use my own maps for forecasting! They’re not meant for that. They’re meant to explain and so there are words and borders and things which get in the way of meteorology but are critical to viewers.

I went to bed for a few hours at 5:30. My server is again churning out weather maps including lightning!

Leave a Reply

Your email address will not be published. Required fields are marked *