SunPy Talk at University of Graz

Florian Mayer has recently given a talk introducing Python and SunPy to students at the University of Graz, Austria. A recording of his talk (in German) and slides (in English) are available online. Thanks Florian!

Posted in meeting, Tutorial | Leave a comment

SunPy 0.1 Released!

We are proud to announce our first release, SunPy 0.1! In this release we have taken the first steps towards a viable data analysis environment for solar physics in Python. SunPy 0.1 can now

  • open and plot fits files from SDO/AIA, SOHO/EIT, TRACE, and RHESSI as well as other solar observatories along with many of the relevant color tables for these data.
  • Query and download data directly from the Virtual Solar Observatory (VSO) as well as query the Heliophysics Event Knowledgebase (HEK).
  • provide basic solar-related physical constants (e.g. solar radius, mass, luminosity) in a quick and convenient way.
  • provide a GUI tool to open fits files and interact with them.

The GSFC SunPy team (Steven Christe, Keith Hughitt, Jack Ireland, Alex Young, and Albert Shih) would like to thank our two SunPy summer students, Florian Mayer and Matt Earnshaw, for their contributions! We look forward to continuing to work with them and would like to encourage the solar community to join us and contribute to the SunPy effort.

Posted in Uncategorized | Leave a comment

SunPy PlotMan: Introduction

Hi, I’m Matt Earnshaw, a first year Physics student at Imperial College London. Currently, I’m working on the SunPy Plot Manager GUI, aka. Plotman.

The goal of Plotman is to provide a convenient interface for manipulating plotted data such as spectrograms, light curves, images etc. making easy the tasks that would be tedious and time consuming to carry out at an interactive prompt. This is achieved by providing instant visual feedback and easy access to the relevant parts of SunPy such as its built in colour maps.

Thus far, I have implemented a tabbed window interface for working with multiple plots and a dockable colour options window that allows switching between any of SunPy’s or matplotlib’s colour maps, clip values selectors and a histogram of the plotted data, as well as integrating all of the plot manipulation functionality provided by matplotlib. At the moment I consider this is all to be very much at the prototype stage.

The interface thus far.

Next I will start work on the over-plotting interface for contours, limbs etc. and providing support for animations using the MapCube object. In the future we may add an interface for downloading data from VSO using Florian’s API, amongst other things.

The latest Plotman code can be found in the plotman branch of my GitHub repo. I am open to any feedback or suggestions that you may have, either in IRC or via the mailing list.

Thanks to the SunPy team for their support and providing the opportunity to work on this great project. I am pleased to be supporting the effort to encourage the use of free and open source software in science, and hope that my contributions will be as valuable to SunPy as the experience of working on the project has been to me so far.

Posted in Uncategorized | Leave a comment

SOCIS progress

Hello! I am Florian, the student by the ESA SOCIS 2011 to work on SunPy. I am currently a student at the Vienna University of Technology in Austria majoring in Software and Information Engineering . The code I am working on can be found on GitHub segfaulthunter / sunpy. In this post I am going to give a short introduction to the work I have done so far. We will be posting short tutorials sometime soon.

First, I have been working on a VSO (Virtual Solar Observatory) adapter in Python (Code here, documentation here) which is now finished. The VSO is a service to obtain solar data from the instruments that observe it. I have conceived a new API that is capable of expressing more complicated queries than the one in IDL was able to; moreover I have added a legacy API that tries to mimic the IDL search query as closely as possible (it does not attempt of mimicing IDL’s feature for expanding substrings of parameters to the respective parameter because the feature does not make sense in the context of Python). I consider the adapter stable and functional by now.

I have also been working on an adapter for the HEK HER (Helioevents Registry). The HER is a service to query for solar metadata (data about events that happened on the Sun). It currently works but needs some more attention for users to be able to get the most out of it. I have also conceived an API for this service that I think naturally exposes it to Python programming language.

It has also been important to me to review and, if necessary, refactor old code, work on unittesting, and help the team to get the hang of git and the other tools used for developing SunPy. In less than a month I will be giving a talk at the University of Graz to introduce scientific computing with Python and SunPy and preparing it will be the main focus of the following weeks to come. If all goes well a recording of the talk will be available for everyone to view online.

At this point, I would like to thank the SunPy team and ESA for giving me the opportunity to work in a scientific environment and I hope that the following months will be as fun and productive as the last month!

Posted in Uncategorized | 1 Comment

Reading in RHESSI image fits files with SunPy

Sunpy can now read in RHESSI image fits files.  To try it out for yourself, grab the latest version of SunPy from GitHub.

Example:

import sunpy
from matplotlib import colors

rhessi = sunpy.make_map(sunpy.RHESSI_IMAGE)
rhessi.show(norm=colors.Normalize(0, 2))

Notes:

  1. The colors module of Matplotlib provides the ability to set the scale of the color table. It is necessary to set this by hand right now but this will be updated soon so that you don’t have to worry about it the first time you open an image.
  2. We now provide convenience variables to access the test files available in SunPy. That means you no longer need to worry about where they are physically located.
  3. Currently SunPy can only read in RHESSI fits files which contain only one image.
Posted in Tutorial | Leave a comment

Initial support added for working with image cubes

Sunpy now has a simple class for working with image cubes called “MapCube.” To try it out for yourself, grab the latest version of SunPy from GitHub.

Example:

import sunpy
from matplotlib import colors, cm

cube = sunpy.make_map('data/AIA2010*', type='cube')

cube[0].show(norm=colors.Normalize(0, 2000))

diff = cube[2] - cube[4]
diff.submap([0, 1000],[-1000,0]).show(cmap=cm.gray, norm=colors.Normalize(-500,500))

output

Notes:

(2) The colors module of Matplotlib provides different scaling methods, and the cm module deals with colormaps.

(4) You can either specify a directory, or a path with wildcards to indicate which files to read in.

(6) cube[n] returns Map object

What’s next?

The existing implementation for MapCube is pretty basic and not very sophisticated. Work needs to be done to normalize the scale and coalign images that do not overlap precisely. Even more interesting would be the creation of subclasses targeting specific types of applications, e.g. ImageSeries, CompositeImage, MultispectralImage, etc.

Posted in Uncategorized | Leave a comment

SunPy Selected for SOCIS

We are proud to announce that SunPy was selected by the European Space Agency (ESA) Summer of Code in Space 2011 (SOCIS) program! This is a program similar to the Google Summer of Code which is meant to support open source software projects. In this case, SOCIS is a program run by the Advanced Concepts Team of the ESA that offers student developers stipends to write code for space-related open source software projects. SunPy was one of several projects which were selected. We received a large number of great applications from prospective SOCIS students and it is unfortunate that we were only able to choose one student. After much deliberation, we are proud to introduce Florian Mayer as the SunPy SOCIS intern! We are looking forward to working with him this summer (and perhaps beyond) and would like to thank ESA for their support!

Posted in Uncategorized | Leave a comment

Python Meeting (5-May-2011)

The slides for Python meeting which occurred at GSFC on 5-May-2011.

Posted in meeting | Leave a comment

Getting started with Eclipse and PyDev

There are many great IDEs available for Python, but one of the most powerful has to be Eclipse (or Aptana Studio, an Eclipse derivative geared towards web development)  with the PyDev plugin installed.

Aptana Studio 3 Screenshot

Aptana Studio 3 In PyDev perspective

Some of Eclipse/PyDev’s features include:

  • Integrated Python console and debugger
  • PyLint integration
  • Auto-completion
  • Cross-platform

The list goes on. One drawback to all of the functionality these tools provide, however, is that figuring out how to use them can be a bit overwhelming for the uninitiated. Indeed, there are a number of books just on using Eclipse! Fortunately though, the most useful features of Eclipse are not hard to get the hang of, so don’t feel like you shouldn’t use Eclipse just because you don’t know what every single feature it has is.

The first thing you need to do is to get started is to install Eclipse and PyDev. The section below describes how to do that. However, an alternative option is to install Aptana Studio 3 which comes pre-installed with PyDev. Aptana Studio 3 includes some additional features as well that are mostly geared towards web development, and also includes a very nice theme system that is not present in the base version of Eclipse. Either solution will work. If you choose to use Aptana Studio skip the first section below and continue at PyLint installation instructions.

Installing Eclipse and PyDev

To begin, download the latest version of Eclipse “Classic” (3.6.2 at the time of writing), and extract the archive to your home directory or some other suitable location. Eclipse is now installed!

Next, open Eclipse and click Help -> Install New Software…

Eclipse update source selection dialog

Adding PyDev to Eclipse software sources

Once the source has been added, check the box next to “PyDev for Eclipse”. Hit “Next” and accept the agreement to add the plugin to Eclipse. It will take a couple minutes now to download and install PyDev. When the installation is finished you will be prompted to restart Eclipse: go ahead and restart it. When Eclipse is first loaded you will be greeted with a Welcome screen with links to tutorials and other information to get you started. For now let’s skip over that and go straight to editing mode. Eclipse allows you to switch between different modes or “perspectives” depending on what you want to do. Each different perspective has its own layout and behavior which are optimized for performing that particular task. For instance there are difference perspectives for working on Java code, for debugging Java, for working on Python, etc. Let’s switch to the Python perspective.

Click Window -> Open Perspective -> Other , and select “PyDev”.

Now we are ready to create a new PyDev project and configure PyDev, but first let’s install a helpful tool that PyDev can make use of: PyLint.

Installing PyLint

PyLint is a helpful tool that scans Python code and finds lines that may either be problematic or break convention (PEP 8). Usually, PyLint is run on the command-line and displays it output there. When configured in PyDev, however, problematic lines are highlighted in your editor making it easy to quickly spot and correct mistakes. If you haven’t already installed PyLint on your computer, now is a good time to do so.

First, download and install setuptools if you haven’t already done so that we can use easy_install to install PyLint. Once you have easy_install (or easy_install.exe on Windows) run:

easy_install pylint

PyLint should now be installed. To make sure it was installed correctly you can try running “pylint -h” from the command-line to show the help documentation.

Creating a new project

Now that PyDev and PyLint have been installed we are ready to create our first project. In Eclipse/Aptana Studio, click File -> New -> Project and in the PyDev section choose “PyDev Project.” Hit Next. Now, choose a name for the project (e.g. “SunPy”), and under “project contents” uncheck the option “Use default” and in the “Directory” field enter the root folder for the code you want to import, e.g. “/home/user/sunpy-dev.” For Grammar choose either 2.7 or 3.0. Uncheck the “Create default ‘src’ folder…”

Before you can finish the new project creation you will need to configure and select an interpreter that PyDev can use when evaluating your code. Click on the link that says “Please configure an interpreter…” to open the relevant PyDev configuration panel.

The Python interpreter can either be configured automatically or manually. In most cases the Auto Config option will work fine and should be used. The only case where you will need to manually add the interpreter is if you have multiple versions of Python installed on your system and Auto Config finds the wrong one. In that case you will need to click “new” and then specify the location of your Python interpreter (e.g. /usr/bin/python2.7). In either case use the default values suggested when choosing which additional packages to include. Once the interpreter has finished setting itself up, hit “Ok” to finish creating the project.

If all went well you should see your project listed on the left-hand panel. Collapse the folder to see the sub-directories and files within your project. The last thing you need to do here is to tell Eclipse to include your code directory in the Python library path. To do this, right-click on the project name in the left panel and go to “Properties.” In the “PyDev – PYTHONPATH” remove the default path that is added (e.g. “/SunPy/src”). Then click “Add source folder” and choose the “sunpy” directory, e.g. “/SunPy/sunpy.”

Hit “Ok” to accept the changes.

Configuring PyDev to use PyLint

The reason we installed PyLint before creating a new project was so that when we created setup the Python interpreter, the PyLint libraries would be included. If we had created the interpreter before creating a project we would have had to go back to the interpreter configuration and manually add PyLint.

To enable PyLint, open the Eclipse/Aptana preferences dialog by clicking Window -> Preferences and navigate to PyDev -> Pylint.

Check the box next to “Use Pylint?”, and enter the location of your “lint.py” which was installed as part of PyLint. The location will vary from system to system, so you may need to search your system for the file (on a Mac the file location might be something like
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pylint-0.23.0-py2.7.egg/pylint/lint.py).

PyLint configuration dialog

Configuring PyDev to use PyLint

Finally, there are two useful arguments which we will tell PyDev to pass to PyLint each time it is executed: the first is “–reports=n” which disables the summary reports that are printed by default each time PyLint is run. The second is “–rcfile=…” Here you should specify the location of the SunPy project rcfile which is located in the the “tools/pylint” folder of the code branch. This will help to ensure that all of the developers working on the project share the same PyLint configuration. Hit “Ok” to accept the changes.

Conclusion

And that’s it! Eclipse and PyDev have a large number of useful features and a multitude of keyboard shortcuts. Take some time to browse through the Eclipse documentation and read other articles to become more familiar with PyDev. It may seem like a lot to take it all at once, but you can learn the features little by little. The first step is simply to get familiar with using Eclipse to edit your code.

Posted in Tutorial | Tagged , , , , | 2 Comments

iPython and Terminal line wrap bug

I’ve been facing a very annoying bug while using iPython in the MacOSX Terminal ever since we started the sunpy project. iPython’s line wrapping in the Terminal would frequently get very screwy such that the history function was almost unusable! The positive outcome of this is that it forced me to try bpython which is also a very competent Python shell. On the other hand, I learned I preferred the functionality provided by iPython compared to bpython. Thankfully I have finally found a solution to my bug! The solution can be found here. It turns out that my readline install (if I ever had one) was not found by iPython so I just had to reinstall it. Using easy_install this is very easy.

easy_install readline

This solved my problem completely.

Posted in Uncategorized | 1 Comment