Space Weather Archive

Many folks may not know that I'm a bit of a fan of space weather. I've spent a considerable amount of my nerdy free time digging through the archives at the Space Weather Prediction Center (formerly the Space Environment Center) website. The SWPC is a division of NOAA, and is responsible for collecting data and monitoring events related to space weather. This includes monitoring changes in the Earth's geomagnetic field, and collecting data about particle and electromagnetic radiation (such as x-rays) that are constantly being hurled at us by the Sun.

The SWPC utilizes data from ground stations, as well as a number of satellites, particularly the GOES satellites, to collect and archive space weather data. This information is then used to assist in predicting space weather, and alerting the world to significant space weather events such geomagnetic or solar radiation storms. The SWPC website includes an education section for those interested in learning all about space weather.

Well, enough about that. In this blog I wanted to write about a web-based application called the Space Weather Archive that I built to assist myself and others in easily accessing historical space weather information. The application is available here, and is free for anyone to use. The archive goes as far back as 1996, primarily because that's when the archive on the SWPC FTP archive begins :)

Usage of the tool is fairly straight forward. Simply input a date and click "Submit". The result will include both geomagnetic and x-ray data, as well as links to the actual data and plots from the SWPC archive. Once caveat is that the data for a particular day will not be available until the following day. This is because the SWPC compiles the data and generates a summary file once per day (for the previous day's data), which we use to retrieve some of our information.

There are also other methods we can use to query the Space Weather Archive using a GET request, and passing arguments so that it returns plain text data. This would be possibly be useful to those interested in querying the archive from within their own applications. But before I get into that, I want to write a bit more about the data from which this data is mined.

The SWPC archive is available via FTP, but probably more conveniently over the web at http://www.swpc.noaa.gov/ftpmenu/. If you explore the archive you can see that there is A LOT of data there. I myself spent many hours in here looking for the correct data I wanted to study. The archive includes both text files with columns of data, as well as plots in .gif format. Older data is stored in what they call the warehouse. Folks wanting to access data more than a year or so old will need to look in here.

Much of the data from previous years is compressed as .tar.gz files, which makes it a bit more difficult to immediately access. They tend to do this around the first week of January. For this reason the Space Weather Archive I built actually pulls data from my own mirror that contains the uncompressed archives. This just makes sense anyway. It's faster and easier on the SWPC website.

So I mentioned earlier that we can also query the Space Weather Archive using just a standard GET request. To do this one must simply pass the year, month and day to the application in the URL:

http://www.aghostonline.org/weather/archive/sec_history.php?format=0&year=2008&month=05&day=04

The format=0 portion tells the Space Weather Archive to output the data in text format, rather than the default HTML. The output of the above URL looks like the following:


Date: 2008-05-04

Geomagnetic Kp: 1:2:3:3:1:2:2:2
Geomagnetic Ap: 8
Geomagnetic Summary: Semi-Quiet
Geomagnetic List: http://www.aghostonline.org/weather/archive/SEC_DATA/2008/SGAS/20080505SGAS.txt
Geomagnetic Plot: http://www.aghostonline.org/weather/archive/SEC_DATA/2008/2008_plots/kp/20080504_kp.gif

Xray (1.0-8.0 Ang): 3.7300e-09 W/m^2
Xray (0.5-3.0 Ang): 5.3916e-09 W/m^2
Xray High: 9.5000e-09 W/m^2
Xray Low: 3.7300e-09 W/m^2
Xray Peak: Normal
Xray Summary: <B Class (Normal)
Xray List: http://www.aghostonline.org/weather/archive/SEC_DATA/2008/2008_XRS_5min/20080504_G10xr_5m.txt
Xray Plot: http://www.aghostonline.org/weather/archive/SEC_DATA/2008/2008_plots/xray/20080504_xray.gif


Thats it! The information includes x-ray and geomagnetic data for that particular day, including links to the actual data files and plots. The output format should make important and parsing into another application relatively easy.

I hope you find this data as interesting as I do. If you do end up using the Space Weather Archive for your own application, please feel free to drop me an email and let me know :)

No comments: