From autoplot.org

1. Introduction

Autoplot is an interactive browser for data on the web. In the same way that you browse content stored in .html, .jpeg, and .png files on the web, Autoplot allows you to interactively browse data stored in .cdf, .nc, ascii, and many more data file formats.

Autoplot tries to create a sensible plot given a URL or a name of a data file on your computer. Axes and plot settings are quite interactive, so you can quickly get your data from data file to publishable graphic within minutes.

To get started, click this link Start Autoplot and then select some of the demo bookmarks found under "Bookmarks" in the menu bar.

2. Installation

Java applications and Autoplot have the goal of being installationless, in that there's a trivial procedure to start the application. Most operating systems come with Java pre-installed and pre-configured so that Autoplot should work immediately on your desktop. The installation instructions below reflect specific aspects of this procedure at this time (Spring 2012). The installation instructions below assume that you have a recent version of Java installed (a version from 2005 or later). You may install Java by going to java.com/en and clicking "Free Java Download".

First click this link Start Autoplot and follow the instructions given below for you browser/operating system combination.

See also help#WebStart

2.1. Windows 7

2.1.1. IE9

Click "Open" (If you do not see the "Open" option, you need to install Java by going to java.com/en and clicking "Free Java Download".)

2.1.2. Mac

People are having problems using Java applications on Mac OS X Lion. Here are notes about installing on this platform.

2.1.3. Chrome

When you see this appear on the bottom of the browser window, Select "Keep"

then click on this

To have Autoplot start in the future without requiring a click on the downloaded jnlp file, select the arrow on the right-hand side of the downloaded file and select "Always open files of this type".

2.1.4. Safari

You will see this pop-up. If Autoplot does not start Automatically, click on autoplot.jnlp.

2.1.5. Firefox

When you see this pop-up window, select OK: (If you do not see the "Java Web Start" as an option in the drop-down list, you need to install Java by going to java.com/en and clicking "Free Java Download".)

2.2. Linux

The default configuration of many modern Linux operating systems uses a non-Sun implementation of WebStart and Java (e.g., IcedTea/OpenJDK), which works poorly with Autoplot, and sometimes not at all. (Autoplot requires standard features of Java that have not yet been implemented in many non-Oracle versions of Java.)

To launch Autoplot from a web browser on Linux, follow these steps:

1. Install Sun/Oracle Java (do not use IcedTea/OpenJDK)
2. Find the location of javaws with the word sun in the path, e.g., /usr/lib/jvm/java-6-sun-1.6.0.24/bin/javaws. (This is usually the first result listed after entering locate "/bin/javaws" | grep sun on the command line.)
3. Click here.
4. When prompted with an "Open With" dialog, enter the full path to javaws (note that on Chrome you will need to click on the downloaded file to get the "Open With" dialog)
5. If you still have problems, see [1].

2.3. Errors

Sometimes updates in Autoplot's dependencies cause this error:

Unsigned application requesting unrestricted access ...

To fix, start javaws -viewer and delete all CDF and Autoplot applications and then delete all CDF and Autoplot resources. Webstart is touchy software and resetting things almost always seems to help.

3. Navigation

3.1. Address Bar

In the address bar at the top of the GUI you can enter the name of a supported format type. If the data address is a URL, http or ftp is used to make the remote file local. A fully-qualified URL will contain both the name of the file and additional arguments to read in the data. For example, it may contain a CDF filename, plus "?BGSM" to indicate the variable to plot. Together these form a URI, which is the precise address of the data. Autoplot will help build URIs by providing graphical editors to pick the data. Autoplot URIs are designed to give a concise way to refer to data: put a data file in a web location, and send the URI to a colleague so he or she can look at it too.

Note Autoplot has a basic and expert mode. In basic mode, the address bar is not visible and is replaced with a controller for the time range.

Sometimes Autoplot URIs do not refer to files. They still locate data, but one of Autoplot's plug-ins interprets the address. For example, vap+cdaweb:ds=CRRES_H0_MEA&id=FLUX&timerange=1991-10-09 points to data from the CRRES instrument provided via CDAWeb at NASA/Goddard. vap+audiosystem:spec=512&len=2.5 samples from the local machine's soundsystem for 2.5 seconds.

3.2. Bookmarks

Bookmarks may be selected in two ways

1. By selecting the Bookmarks menu item and then browsing a tree of bookmarks.
2. By selecting Bookmarks->Browse and Manage and clicking on a bookmark.

Note that the section option allows you to view metadata associated with a bookmark (if available) by clicking on the Detailed Description button. In addition, the second option allows you to place a plot for a given bookmark in a separate panel or on top of the current panel in view.

3.3. GUI

Basic user interactions including: Left-click and drag to zoom, Middle-click and drag to pan, Click on axis and drag to constrain interaction to one axis, Ctrl-Z / Ctrl-Y to undo/redo changes (CMD-Z/CMD-Y on Mac), Shift-Click on plot to move plot box.

More complex interaction with plots, including: mouse wheel control, keyboard entry of axis range, context overview plots

Here are more videos, described below: www.youtube.com/user/AutoplotDevelopers

3.4. Keyboard

Keyboard shortcuts.

• ctrl-Z undo ctrl-Y redo
• ctrl-R reset zoom
• ctrl-shift-R reset Y zoom
• ctrl-shift-X reset X zoom
• ctrl-shift-Y reset Y zoom
• ctrl-plus increase font size
• ctrl-equals increase font size
• ctrl-minus decrease font size
• ctrl-tab next plot element
• ctrl-shift-tab prev plot element

3.4.1. General

• CTRL + play button (green arrow button) adds a plot below.
• Shift + play button (green arrow button) adds an overplot.
• CTRL + R resets the zoom of the plot in focus.
• TAB while typing in address bar to show list of completions.
  • Example: autoplot.org/data/ + TAB will show a list of files in that directory
  • Example: autoplot.org/data + TAB will only show a list of files that you have accessed previously that match the string

3.4.2. Axes Shortcuts

When pointer is over an axis

• Middle mouse button drags the axis.
• Mouse Wheel zooms in and out.
• Ctrl + Mouse wheel pans the axis. This allows for rapid scanning of the data.

4. Tabs

4.1. Axes

The Axis tab contains basic controls for the two axes of the plot in focus, the plot title, and the colorbar title (the colorbar may not be visible but it always exists).

Ranges for the x-, y-, and colorbar are strings that are parsed using the current axis units (such as time or dimensionless). For example,

• For an axes with time units: "2000 through 2010", "Sept 16 2009", "2009-009 10:00 to 11:00", and "2009-009T00:00 to 2009-009T10:00" "2012-12-14T23:00/2012-12-14T24:00".
• For physical ranges, use the keyword "to" to delimit two real numbers. When the axis has a physical unit, such as "m/s" the range can be qualified with a unit, as in "0 to 100 cm/s."
• An automatic test showing example ranges is here.

All labels may include "Granny" strings defined by Grandle and Nystrom [2]. A few of these are

•  !A superscript
•  !B subscript
•  !E superscript with smaller font size
•  !D subscript with smaller font size
•  !N normal position
•  !C carriage return (newline)

so that !A2!Ns!A-2!N results in m²s^-2.

Labels may include Unicode (indicated using its entity number, e.g., Å). In addition, the named html entities listed at [3] and [4] may be used. The list includes:

• Greek letters: Α Β Δ α β δ π ρ ω
• Math symbols: ∑ (∑) ± (±)

When selected, Isotropic constrains the x- and y-axis to have the same data-to-pixel scaling. This option is ignored when the axes do not have compatible units.

When Legend Label is checked, a legend is added to the plot, and an icon for this panel is added.

4.2. Metadata

The Metadata Tab provides information about the data. It is a tree view that contains a number of nodes of information.

The Metadata node is metadata provided by the data source. In general, Autoplot doesn't understand the metadata and simply provides it for reference by the user. However, if the data source also identifies a "metadata model," this is indicated in parenthesis and Autoplot will use the metadata to get axis labels and ranges.

The Statistics node provides basic statistics about the data. For example it shows the mean and standard deviation, and the number of valid points. Cadence indicates the typical spacing between points and is used to insert breaks in time series.

The Dataset node allows some inspection of the dataset itsself. The size of the dataset, its named dimensions, and their size are identified. Metadata for plotting found within the dataset is available as well. You can look at the digital data (though only the first twenty or so points) as well.

4.3. Style

The Style tab contains controls for the style of the plotted element.

Autoplot tries to pick reasonable default settings such as data rendering method (series versus spectrogram), but also default symbol sizes and connectors. To further specify how a plot should appear, click on the plot element to modify its color or symbol, for example, or for spectrograms the colorbar (e.g. grayscale vs colors).

Note that you can set the rendering method and default settings via the plot context menu (right-click), under Plot Style.

Autoplot will swap out controls based on the rendering method.

5. Adding Plots

Right click->Add plot can be used to add additional plot on the plot. Axes from multiple plots can be bound together so that adjusting one triggers a change in others. This video shows how to add a second plot and bind the y-axes together. Note Autoplot tries to bind time axes together automatically, since this is a common mode of operation. (A future version of Autoplot will allow redundant x-axis labels to be removed to save space when many plots are used.)

5.1. Layout

The Layout Panel (enable with Options->Enable Feature->Layout Tab) provides access to all of the plots and plot elements on the canvas. This allows you to swap plot position, delete plot elements, add groups of plots, etc.

5.1.1. Plots

The Plots area contains a simplified view of the plots on the page. Each plot has a context menu that allows the plot to be edited or deleted. Also a group of plots can be added below, for example a block of 3 by 3 plots.

Multiple plots can be selected by holding the control button down while clicking. Two plots can be selected and Plots->Swap Position will exchange their positions. Also multiple plots can be edited at once, for example to set all the titles the same and then make tweaks for each plot.

5.1.2. Bindings

This Bindings area contains a list of bindings for the application. These are properties that are bound together. For example, a stack of plots with a common x axis all have their x axes' range bound to the DOM property "app_0.timerange". If one right-clicks on a binding, it can be deleted.

5.1.3. Plot Elements

Plot elements have the job of rendering the data onto a plot. The Plot Elements section allows them to be individually controlled and deleted. Note when a vector is plotted (e.g. Bookmarks->Demo 5), then a plot element is added for each component, and these are all children of an inactive parent component. The inactive component can be used to control all the children at once. For example, adjust the line thickness of the parent and all the children will be modified.

5.2. Data

The Data Panel (enable with Options->Enable Feature->Data Tab) contains controls for processing the dataset after it is loaded.

For example, the fill value can be set manually when the dataset has failed to identify it.

The Data Source sub panel controls the data source node that has focus. The URI of the data is indicated for convenience in a small font. Valid Range explicitly overrides the valid range of the dataset.

Data Post Processing specifies the operations that should applied before displaying the data. Originally introduced to allow for slicing of high-dimensional (high "rank") data sets, this also has a number of operators that are described below.

When the Operations field is "|slice<dim>(<index>)" the slice dimension and transpose gui controls become enabled to provide a more usable control for the slice. For example, turning the mouse wheel over the slice index control or operations text field will rapidly adjust the slice position. (Undock the data tab to see the slices as the adjustment is made.) Slice dimension controls which dimension should be sliced. Note that the plot element will automatically set this when a high dimensional dataset is plotted. Transpose will flip the dimensions of the data before plotting.

When a vector component is plotted, component simply identifies the component to plot.

Additional Operators When the operations field starts with the pipe (|) character, it is a list of filters that are to be applied to the data. These filters include:

|histogram perform an "auto" histogram of the data that automatically sets bins.

|logHistogram perform the auto histogram in the log space.

|log10 take the base-10 log of the data.

|exp10 plot pow(10,ds)

|slice0(index) slice the data on the zeroth dimension (often time) at the given index.

|slice1(index) slice the data on the first dimension at the given index.

|collapse0 average over the zeroth dimension to reduce the dimensionality.

|collapse1 average over the first dimension to reduce the dimensionality.

|transpose transpose the rank 2 dataset.

|fftWindow(size) plot power spectrum by breaking waveform data in windows of length size (experimental, not for publication).

|fftPower(size) plot power spectrum by breaking waveform data in windows of length size (experimental, not for publication).

|hanning(size) apply hanning filter before fftPower.

|smooth(size) boxcar average over the rank 1 data.

|diff finite differences between adjacent elements in the rank 1 data.

|accum running sum of the rank 1 data. (opposite of diff).

|grid grid the rank2 buckshot but gridded data into a rank 2 table.

|flatten flatten a rank 2 dataset. The result is a n,3 dataset of [x,y,z]. (opposite of grid)

|negate flip the sign on the data.

|cos cos of the data in radians. (No units check)

|sin sin of the data in radians. (No units check)

|toDegrees convert the data to degrees. (No units check)

|toRadians convert the data to radians. (No units check)

These filters can be chained together like so: |slice1(10)|histogram. The GUI in Autoplot2011 contains a popup list for convenience. Trigger a popup on the editor to see the list. The final version of Autoplot2011 will have a GUI for editing this string. In the DOM, this is the same as the "component" property of a plotElement.

5.3. Script

The script panel (enable with Options->Enable Feature->Script Panel) provides a convenient place for writing and reviewing the Jython scripts to control the application and load data. It provides a simple completion and right-click to see additional options. This is intended to provide full access to advanced users.

Note a .jyds file executed in the "data source context" and loads data from a number of places and performs operations to create a new dataset. .jy files are executed in the "application context" and control labels, add components, run batches and plot ad-hoc datasets.

Another use case for scripting is to add new functionality to the GUI. Scripts that run in the application context can be put in the folder "HOME/autoplot_data/tools/", and will appear under the tools menu when the app is reloaded (or getViewWindow().reloadTools() is run). For example, save this in the file "HOME/autoplot_data/tools/flashFocus.jy:

# label: Flash Focus
#cause the selected plot element to flash

r= java.util.Collections.singletonList( dom.controller.plotElement )
for i in range(3):
  dom.controller.canvas.controller.indicateSelection( r )
  java.lang.Thread.sleep(500)

Reload the tools, and a new menu item "Flash Focus" should appear. The script will cause the focus plotElement to flash three times. Note the URI "script:autoplot.org/data/tools/flashFocus.jy" will display a verification GUI that also allows the script to be installed in the tools area.

Jython scripting is being finalized and documented. More documentation is coming soon.

6. Cookbook

The cookbook page describes quite a few useful tricks, such as resizing plots and putting exponents in labels.

7. Advanced Topics

7.1. Layout fine-tuning

DOM GUI (Accessed via Edit->Edit DOM) showing entries for layout. Autolayout must be de-activated for changes to apply (Options->Auto->Autolayout)

Autoplot's canvas consists of a number of plots which are the two-dimensional spaces on which data is rendered. Plot elements connect data to the plots, by identifying the data source, plot, and method for rendering data. A plot can contain a number of plot elements, as is the case when two datasets are drawn on the same axes, (also referred to as an overplot.) A color bar is connected to all plots so that multiple spectrograms can be rendered using the same color map. The color bar is hidden when it is not needed.

The cookbook page describes quite a few useful tricks, such as resizing plots and putting exponents in labels.

Autoplot uses rows and columns to specify the location of components on the drawing canvas. These objects are controlled with strings that efficiently specify the location. For example, the horizontal (column) layout from the left is specified as a percentage plus a shift from the left L%+S, where

• L is the percentage of canvas to place the left side of the plot box (only data, no labels)
• S is the shift (in px or em)

To have the labels on the y-axis rendered outside of the canvas, use 0%+0

The layout may be specified in the URI sent to the Autoplot server. In this case, the left and right positions are comma separated, as in L%+S,R%+S. The default is 0%+8em,100%-6em. For the bottom and top positions, the default is 100%-3em,0%+2em.

Note that rows and columns are children of the marginRow and marginColumn. Instead of specifying the normal position with respect to the canvas, these rows and columns position are specified with respect to the marginRow and marginColumn. The autolayout feature automatically adjusts the position of the marginRow and marginColumn to make the plot fit on the page.

7.2. Time Parsing / Formatting

One code for parsing and formatting times into strings is used throughout Autoplot to:

• parse times within a file
• parse times in a filename (and URI)
• generated times in scripts

Note this is different than the code that parses ranges to control axes. That one must guess the format, then parse. This one is given a time format, and then and parse strings to get timeranges or vice-versa.

Example times:

• $Y-$m-$d 2010-06-23
• file:///tmp/$Y$m$d.dat file:///tmp/20100623.dat

Field types (see #Wildcard_codes for full list and definitions):

• $Y $m $d $H $M $S $(milli) $(micro)

Field qualifiers:

• $(H;span=4) digit indicates beginning of 4 hour interval
• $(m;Y=2004) The year 2004 is implicit

Note %{m;Y=2004} may be used instead

7.3. Aggregation

Data coming from files can typically be combined, or aggregated. For example, a single CDF file can be read (using the CDF plug-in) with the URI:

 vap+cdf:ftp://cdaweb.gsfc.nasa.gov/pub/istp/ace/swe/2008/ac_k0_swe_20080620_v01.cdf?Np

If this is one of a series of files, the variable Np can be plotted across several files using the notation:

 vap+cdf:ftp://cdaweb.gsfc.nasa.gov/pub/istp/ace/swe/$Y/ac_k0_swe_$Y$m$d_v$v.cdf?Np&timerange=2008-June

Most of the letters used for the wildcards for time are the same as the Unix date command, for example $Y means a four-letter year while $y means a two-letter year.

Any file-based reader that returns time for the first dimension can be aggregated. Note "Tools->Aggregate..." will attempt to create an aggregation spec from a file URL.

7.3.1. Aggregation GUI

Aggregation URIs will have an additional controller in the data source editor. This allows timeranges to be browsed and selected. The field "Time Range for Aggregation" controls the span that will be aggregated. This is a time range parsed like other timeranges in Autoplot: 2010 is a year, 2010-Jan is a month, 2010-Jan-01 is a day, 2010-Jan-01 07:00 to 08:00, and so on. Below are three droplists that allow an interval to be selected. The GUI will query the server to see which intervals are available. First select from the available years, then from available months within that year, and then a day. "Copy" will then copy the value into the time range text field. The reduce checkbox enables the experimental data reduction feature where data is reduced to axis resolution as it is read in.

7.3.2. Wildcard codes

Aggregation URIs will contain from the following wildcard codes:

• $Y four-digit year
• $y two-digit year
• $j three-digit day of year
• $m two-digit month
• $b three-letter month (English locale only)
• $d two-digit day of month
• $H two-digit hour
• $M two-digit minute
• $S two-digit second
• $(milli) three digit milliseconds since second boundary
• $(micro) three digit microseconds since millisecond boundary
• $v version number, decimal sort. $(v,alpha) is for alpha sort, and $(v,sep) for "x.y.z"
• $o orbit number, arguments like $(o,id=crres) make this useful. see www.autoplot.org/developer.orbitTimeSpec#Orbits_in_Time_Ranges

A version wildcard is allowed. Versioned URIs have the form:

 vap+cdf:ftp://cdaweb.gsfc.nasa.gov/pub/istp/ace/swe/$Y/ac_k0_swe_$Y$m$d_v$v.cdf?Np&timerange=2011-June

Also, parenthesis can be used to modify fields. For example:

• $Y$m$d-$(H,span=12) indicates the hour is the beginning of a 12 hour interval indicated by the name.
• $(m,Y=2011)$d indicates the year field is implicit. TODO: bugs prevent this from working, since we look for $Y to detect aggregations...

7.4. ASCII Editor

Using the ASCII Table Data Source Editor GUI

Details on how the ASCII table reader is implemented is described at ascii_data_source.

A GUI editor can be used to make it easier to read in data. A table is displayed showing how the reader would parse the file. Darker lines that are not broken into columns are lines identified as non-records, such as header lines.

7.4.1. Header tab: identify records and non-records

Skip Lines: the number of lines to skip before attempting to parse records. Press "Select" then click on the first parseable line.
Comment Prefix: lines starting with this are treated as non-records.

7.4.2. Times tab: specify how times are parsed

Time Format: format for interpreting times. For example $Y+$m+$d means the first three fields are interpreted as year, mo