Data Visualization

1月 102011
 
Bubble plots in JMP are a great way to see how values change over time. But they also allow for the viewer to see so many variables (or dimensions) at one glance. So they can be a valuable visualization tool to view data without a time dimension. Think of that dimension as a page dimension for a book of reports -- or in this case, a set of graphs.

In the example below, which uses performance results by product, I selected the Distribution Channel in place of my time dimension. I created four quadrants and can see the other variables as shown below:



Now, when I click on the Step button, I can page through all my channels and see the results by channel. You'll see three "pages" of bubble plots.







So the bubble plot is not only a way to show changes in values over time. You can use it in very creative ways to show data over any variable.

Can you think of other examples where you could use the bubble plot in this way?
12月 202010
 
This blog post is the final one in a series about image-based background maps in JMP 9.


Part I of the series introduced background maps, which are a new feature in JMP. It explained why background maps are useful and showed how to access a background map through the user interface.

Part II of the series discussed Simple Earth and Detailed Earth. These options are the background maps that are installed as part of the JMP product. Because they are local to the machine, they don't require any Internet connection or rely on server availability. But they have a limited resolution, and the quality of the imagery starts to degrade when you zoom in too close.

Part III of the series discussed the Web-based options for background maps: NASA and Web Map Services. These options support the WMS api and let you retrieve maps from a WMS server on the Internet. A WMS server can often generate a map at a higher resolution, allowing you to zoom in on the graph.

Part IV of the series discussed a JMP add-in called the WMS Explorer. This add-in helps to find WMS servers that are available online. It also gives a preview of what locations the WMS server covers and what types of maps it can return.



I invite you to read these blog posts if you haven't already. I hope they will answer any questions you have about the image-based background maps in JMP 9. I'd also like to point out that others at JMP have written blog posts about the vector-based background maps.

One thing I haven't discussed yet is how to access the background map functionality through JSL. You can find an example of this in the Samples Data directory. Run JMP and open the sample data file called Hurricanes.jmp. Edit the script called Bubble Plot with Map. Look for the command in the script that begins with Background Map.

To figure out the correct parameters for the background map command, simply look at the background map dialog. The wording on the dialog is the same as the JSL commands. To create a background map, simply use the command Background Map().

There are two types of background maps you can specify: Images() and Boundaries(). Each of these then takes a parameter, which is the name of the map to use. The name is one of the maps listed in the dialog. For Images(), the choices are "Simple Earth," "Detailed Earth," "NASA" or "Web Map Service." If you elect to use "Web Map Service," then there are two additional parameters: the WMS URL and the layer supported by the WMS server. For Boundaries(), the choices vary since boundaries can be user-defined. But a typical choice might be "World Countries."

So if I wanted to add a background map that used the "Detailed Earth" imagery and the "World Countries" as a boundary, the command would look like this:

  Background Map ( Images ( "Detailed Earth" ), Boundaries ( "World Countries" ) )

If I wanted to change that to use a WMS server, then the command might look like this:

  Background Map (
     Images ( "Web Map Service", "http://wms.jpl.nasa.gov/wms.cgi", "BMNG" ),
     Boundaries ( "World Countries" )
  )

Note the spacing doesn't matter. I just did it for readability. And here is what the graph would look like:



Tip: Of course, you can always create your graph and add a background map through the user interface. Then from the little red triangle (lrt) use the Script->Save Script to Script Window menu item to see the JSL that gets generated. I find this technique very helpful for learning JSL in general.

So what else can you do with background maps in JMP? Well, apparently you can use them to create the company holiday card. Check it out! Whatever holiday you choose to celebrate at this time of year, I'd like to wish you a happy one. And I hope you have a happy, healthy and successful 2011.

Peace!
12月 132010
 
Graph Builder strives to help you see your data without adding any interpretation. That's why the default view shows the actual data points and the summary element is a spline smoother instead of a regression line. However, even these elements bring some baggage. Data markers have over-striking issues when many are present, and a million of them can take several seconds to draw. The smoother shows an average trend, whether it's representative or not.

Looking at last year's power data from the SAS Solar Farm, the default Graph Builder view is a smoother. Since there are 50,000 data points, the Marker element is turned off by default.



The curve looks reasonable, but without the markers we can't tell if there's more to the story. Turning on markers, we get:



This illustrates the over-striking issue and doesn't tell us anything unless we notice that the scale has changed. One way to help with over-striking is to set the transparency of the markers to something like 0.1.



That's a big improvement, though the bottom markers seem maxed out and hard to compare. Plus, transparency doesn't solve the speed issue. For 50,000 points, JMP is still responsive. But at 500,000 points, it would start trying your patience.

A new alternative for overcoming these issues is the Contour element, which, in the absence of a color variable, will show shaded contours representing point density.



If you squint your eyes, it looks a lot like the view of markers with transparency, but it doesn't have the over-striking or performance issues. With either view, you get the sense that the smoother doesn't go through the densest part of the data, which suggests there are other factors involved. In this case, we might conclude that there are some sunny days and some cloudy days but not many in the middle.

I created a dummy variable called "Sunny" and assigned it to the Overlay role just to illustrate that the density contours can be overlaid.



That doesn't tell the whole story, but we can tell we're getting closer. (We know from previous analysis that time of year makes a difference, too.)

The density contour may look strange if you're not expecting it, which is why it's turned off by default. However, there are new preferences for Graph Builder in JMP 9 in which you can not only control the default elements but also the cut-offs for when Graph Builder switches to the summary view.



For instance, you can set the Continuous Alternate to Contour if you want to see the density contour whenever the Points Limit is exceeded.
12月 012010
 
7,000 -- that's the big number everyone was talking about last week in Paris. 7,000 what?

7,000 Euro/m², that’s the average price prospective buyers now have to pay for properties in Paris. If you are not used to m², 1 m² is roughly equal to 10.8 ft², and whichever way you look at it, that’s expensive.

However, this number is an average. As Kaiser Fung, a speaker at the Discovery Summit earlier this year, wrote in his book Numbers Rule Your World: "Averaging stamps out diversity, reducing anything to its simplest terms. In doing so, we run the risk of oversimplifying, of forgetting the variations around the average.”

Paris is divided in 20 districts, known as “arrondissements.” These are laid out in a spiral or snail shape from the 1st in the center to the 20th, a bit further out, reflecting how the city developed over the centuries. Using the new mapping capabilities of JMP 9, I could very quickly create a map of Paris and use it to go beyond the newspaper headline.

In a recent post, Justin Mosiman explained how to use the Custom Map Creator, a JMP add-in, to build your own map. Just a few minutes after downloading Justin's add-in, I was able to turn data into information and could see beyond the average.

What is quickly apparent is that property prices vary from arrondissement to arrondissement. Not only can we see that the center is more expensive than the edge (not that surprising), but we also see that the east tends to be more affordable than the west. In addition, the 6th district (near Notre Dame Cathedral) is the most expensive district.



So are property prices going through the roof (literally) everywhere? Well, actually, in most neighborhoods the answer is yes since prices went up about 4 percent in only three months. Looking at both maps, you see that the most expensive district (the 6th) is also where prices are rising the fastest. On the other hand, Paris’ most central area (the 1st), has seen a 1.8 percent quarterly drop.



Anyway, let’s just hope that the big number -- 7,000 -- does not get any bigger!
11月 302010
 
Many of you are already aware of the new powerful mapping capabilities in JMP 9 that enable you to visualize geographic-based data without having to provide any longitude/latitude or shape information. For example, I can open a data table in JMP that has two columns – the names (or abbreviations) of US states in one column, crime rate in another – and Graph Builder will draw a map of the US for me coloring the states by the value of crime rate.




This is possible because JMP 9 automatically comes with the shape (.shp) and database (.dbf) files for US states as well as many other geographic shapes such as countries, US counties and provinces of various European countries.

As a JMP Systems Engineer, I often demonstrate this new capability using the crime rate example above. Folks get excited and usually ask which other shapes they can map out. I am happy to tell them about the others that are provided and that they can also add their own shape files very easily to map out whatever shapes they want.

Unexpectedly, I had to prove this claim on a customer phone call recently! This customer was interested in the same mapping capability for US Metropolitan Statistical Areas (MSAs). As I was explaining how easy it would be to find the .shp/.dbf files and add them to the appropriate directory, one of the customers on the phone call found the correct files on the US Census website.

“Hey, Mike, I think I have the files right now, so why don’t you give it a shot?” the enthusiastic customer said. I think we were all amazed, myself included, at how simple this really was:

• I opened the .shp and .dbf files obtained from the Census website in JMP.
• I verified that Shape ID was the first column in the shp file and that it contained X and Y columns (which represent the longitude and latitude coordinates).
• I saved this .shp file to the Maps folder as “US-MSA-XY.jmp.”
• I created a Shape ID column in the .dbf file, which is a column that simply mimics the row number. A nice trick to do this easily: Visit Column Info in a new column and choose Initialize Data>Sequence Data. I moved this column to be the first one in the table.
• Then I assigned the Map Role column property to the name of the MSAs. This is how JMP knows to draw a map when the user provides the name. I right-clicked the CBSA column and chose Column Properties>Map Role and then chose Shape Name Definition from the ensuing drop-down box.
• I saved this dbf file to the Maps folder as “US-MSA-Name.jmp”

That’s it! I reopened the “Name” file I had just saved and dragged CBSA into the shape zone in Graph Builder to verify that the shape name was correctly identified by JMP. So, not 10 minutes after I mentioned it as a possibility, we were viewing an MSA-level map in Graph Builder. After much applause (in my head), these customers were visualizing their MSA data.




So Google “esri shape” and the name of the shape you are looking for, and start adding your own! And keep your eyes open for another blog post about using this method to analyze Afghanistan Province-level data.

The JMP directory that contains these files can be found at the following paths:

Windows:
C:\Program Files\SAS\JMP\9\Maps

Mac:
/Library/Application Support/JMP/9/Maps
11月 222010
 
In my previous blog post, I talked about using a Web Map Service (WMS) to generate a background map. The WMS feature is more flexible than the static background maps provided as part of JMP. But it requires the user to find an appropriate WMS server and then determine specific information about the server, such as layer names.

This can be a time-consuming task in itself. Searching the Web will help you find WMS servers that are publicly available. But it can be more difficult to figure out what the layer names are and what the resultant map will look like. To help with this discovery process, I have taken advantage of another JMP 9 feature called the add-in. An add-in is a way to extend the functionality of JMP. Add-ins (including this one) can be downloaded from the add-ins page on the JMP File Exchange.

I created an add-in called the WMS Explorer. The WMS Explorer add-in is designed to make it easier for you to find a WMS server, query the server for layer names and show you what a map will look like. Here is what the WMS Explorer add-in looks like.

WMS Explorer add-in

At the top of the window is a text field where you can type in a URL to a WMS server. You can find possible WMS servers by searching the Web. After you enter a URL, either hit the enter key or click on the Get Layers button. This will send a query to the WMS server and return a list of layers that the server supports. If you do not know the URL for a server, you can select one from the drop-down list, directly below the text field. Selecting a URL from the list will populate the text field and submit the query for layer names.

The graph, below the drop-down list, will now contain a boundary map. Initially, the graph will show the Earth, with each country outlined. To the left of the graph is a list of layer names that the server supports. Select a layer name, and another request will be made to the server. This request will return a map that will be set as the background map on the graph. Here is an example where I selected the first URL in the drop-down list, the NASA server, and chose BMNG as the layer name.

Example using NASA server and BMNG as layer name

You can select the different layer names to see what each map will look like.

The graph is a typical graph in JMP, which means that all the regular JMP controls are available to you. You can adjust the axes or use the zoom tool (found on the hidden menu bar) just as you would in JMP. You can also right-mouse-button (RMB) to select Size/Scale->Size to Isometric to get your graph back into a proper aspect ratio. RMB will also allow you to select Background Map, where you can adjust the boundary map. Here is an example where I entered a URL for a WMS server in Spain and zoomed in on the Strait of Gibraltar. In this map, Spain is to the north and Morocco is to the south.

Example using WMS server in Spain, creating map zoomed in on Strait of Gibralter

Occasionally, a request to a WMS server will result in an error. At the bottom of the add-in window, there is an area for any error messages. You may try a server that no longer exists or is temporarily down. Or you may select a layer name that the server no longer supports. In a case like this, instead of returning a map, the server may return an error message that will be displayed in this area. Any error message that is displayed is a message that came directly from the WMS server.

Once you find a map that you want to use, simply note the URL in the text field and the name of the selected layer. This is the information you will need to use the WMS background map feature in JMP.

I hope you enjoy the WMS Explorer add-in and find it useful. If you find a good WMS server, please send it to me. I’ll try to keep an up-to-date list available to share with other JMP users. As always, feel free to share your comments and suggestions.
11月 192010
 
I started to work at JMP this past summer as a student intern. One of my first projects was to create a background map for the office temperatures study. As you probably know, JMP 9 can graph map shapes using built-in map files. JMP can also work from user-created shapes, which can be imported from ERSI SHP files or created from scratch. To make the latter process easier, I created the visual Custom Map Creator add-in.

Shapes in JMP are defined by two different data tables that work together, a name file and an XY file. The name file contains the ID and name of the shape that you are defining. The XY file contains four different columns that define the shape coordinates. The first column is the ID of the shape that is defined in the name file. There is also a part ID, which is used if you have multiple parts to a single shape. For example, if you were defining Hawaii, each island would be a part, and all of the islands would have the same shape ID. Finally, the X and Y columns specify the location of one particular point in the boundary.

In the image below, shape 50 has six rows that correspond to the six points. Since the shape is not broken into multiple pieces, it has one part.

a shape that has 6 rows and 6 points

The process of creating the shapes by hand is tedious at best, but the Custom Shape Creator add-in allows you to create the different shape boundaries with the click of a button. All you have to do is drag the image that you want to model into the Custom Map Creator window and use your mouse to define each boundary.

The image shown below is the start of my recreation of the offices map. The Custom Map Creator has a few different visualization options to help you make your boundaries. The line width and color can be changed, in addition to being able to fill the shapes in with a random color (as enabled in the picture below).



I recreated the office map in about 25 minutes with this add-in. No more manually entering in coordinates! As an added bonus, the magnifying glass still works so your map can be really detailed.



The Custom Map Creator is available on the JMP File Exchange. If you have any awesome maps that you created and would like to share I’d love to see them in the comments.
11月 102010
 
Previously, I wrote about the image-based background maps that are provided as part of the JMP install. These maps work well when you are viewing a larger area of the world, such as a country or even a state or province.

But if you want to zoom in closer, these maps start to degrade. The resolution of the Simple Earth and Detailed Earth maps are not high enough to zoom in on a city, for example. To get that level of detail, we need to use another source -- the Internet. JMP 9 has two options on the background map dialog for Internet sources. Those options are NASA server and Web Map Service.

The NASA server provides maps for the entire Earth. Here is a graph of the Earth, which we’ve seen before, but this time it is using the NASA server as its source for the background map. It also has a boundary map turned on, to show the outlines of the countries.



Not only does this server cover the entire Earth, but you can also zoom in on a much smaller area of the Earth and still get a reasonable map. Here is a map showing the Colorado River running through the Grand Canyon, in Arizona. The Grand Canyon Village is visible in the bottom of the map.



If you look at the axes values, you can see that we are zoomed in such that we are viewing an area that is less than 1/10° by 1/10°. The Simple Earth and Detailed Earth background maps cannot display that kind of resolution. The NASA server can give us a fairly detailed view of any land mass on Earth. Water, however, is simply filled in as black. The NASA server is free to access, but it is also limited in availability. If the server is temporarily unavailable or becomes overloaded with requests, it will deliver an error message instead of the requested map.

Another Internet-based option for background maps is a Web Map Service (WMS). The WMS option allows you to specify any server that supports the WMS interface. The NASA server is an example of a WMS server, but we have provided the URL and a layer name for you. With the WMS option, you must know the URL to the WMS server and a layer name supported by the server. Most WMS servers support multiple layers. For example, one layer may show terrain, another layer may show roads, and still another layer may include water, such as rivers and lakes. By specifying the URL for the server and the layer, JMP can make a request to the server and then display the map that is returned.

Here is a popular map around my office. I used a WMS server that provides maps for Wake County, North Carolina. The server URL is http://columbo.nrlssc.navy.mil/ogcwms/servlet/WMSServlet/Wake_County_NC_Maps.wms and the layer is named 16:0. This map plots the location of Starbucks coffee shops around SAS headquarters in Cary, NC. Blue indicates a free-standing Starbucks. Red indicates it is inside another establishment, such as a grocery store or gas station.



There are 20 Starbucks shops within a three-mile radius of my office. It is no wonder that Cary is consistently rated as one of the top cities to live in, in the United States. Perhaps I should use JMP to analyze the traffic patterns between each Starbucks and the office each weekday morning -- and then see how that relates to productivity!

If you want to use the WMS option for a background map, the question then becomes: Which WMS server can I use? There are many WMS servers freely available across the Internet. Most of them provide maps only for a particular area of the world, and each of them supports their own layers. So each user will likely have to search out the appropriate WMS server for his or her particular situation. A good starting point is Google (or whatever your favorite search engine is). Also, here are some links that I have found useful for finding a WMS server:

Once you find a WMS server, you still have to figure out what layers it supports. This can be a little tricky. But never fear -- I wrote a JMP add-in to help you with that. But that is a topic for my next blog post.

Until then, see if you can find a good WMS server for your particular needs. And, as always, feel free to let me know what you think.

All About Background Maps in JMP 9: Simple Earth and Detailed Earth

 Background Maps, Business Visualization, Data Visualization, JMP 9, Tips and Tricks  All About Background Maps in JMP 9: Simple Earth and Detailed Earth已关闭评论
11月 012010
 
In my previous blog post, I gave an overview of the different types of background maps you can add to a graph in JMP 9. Today, I will discuss two of the options for a background map: Simple Earth and Detailed Earth.

Simple Earth and Detailed Earth are two of the choices under Images on the background map dialog. They both consist of a collection of raster images that can be tiled together to make a map of the entire Earth. Both sets of imagery are bundled with JMP and are installed as part of the JMP installation process. This means that no Internet connection is necessary to use a background map -- nor will you have to find and install an extra download.

Here is a graph that uses Simple Earth as the background map.

graph that uses Simple Earth as the background map

To create the tiles for Simple Earth, my source data was Natural Earth III, from www.shadedrelief.com. The Natural Earth III data is public domain and free to use, modify and disseminate. The Natural Earth III texture maps come in different versions from the deluxe edition, which includes clouds and arctic ice, down to a basic version. I opted for a version in the middle, which included better geography but did not include either the clouds or arctic ice. It uses a green and brown color scheme for the land and a constant deep blue for water.

The image that I started with had a resolution of 16,200 x 8,100 pixels. I first increased the compression of the jpeg image, which minimally reduced the visual quality while reducing the overall size of the image (and the resultant tiles) considerably. I then wrote a script that would read in the complete image and tile it into many smaller pieces. I generated three different sets of images.

For the first set of images, each image covers an area that is 10° by 10° at a resolution of 450 x 450 pixels. The images in the second set cover an area of 30° by 30° and are at a resolution of 360 x 360 pixels. Each image in the final set covers an area of 45° by 45°, at a resolution of 540 x 540 pixels. If you calculate the pixel-to-degree ratio for each set, you’ll see that the resolution decreases with each set. The reason for the multiple sets is so that a higher resolution is used when you are zoomed in, showing the highest-quality possible. When you are zoomed out, using a lower resolution reduces the number pixels that have to be processed at render time, allowing the graph to render more quickly.

Here is an example of Detailed Earth used as the background map. This graph might look familiar as it is the same data I showed in my previous blog post. Because I used the Detailed Earth background, we now see more detail in the water. Simple Earth doesn't show any detail in water features.

example of Detailed Earth used as the background map

The source for the Detailed Earth was Natural Earth data. This data also comes in different versions and resolutions. Since I wanted this background map to be more detailed, I started with the raster version of the highest resolution I could get, which was the large scale data, 1:10m. From the different options, I chose Natural Earth 1, with Shaded Relief, Water and Drainage. This version had a softer color scheme than Simple Earth, lighter greens and browns for the land, as well as variation in the blue for the water.

Detailed Earth is so named because it has slightly more detail than Simple Earth, in terms of resolution as well as in the geography and the water features. I created the Detailed Earth tiles in a similar way to the Simple Earth tiles. Like the Simple Earth data, I generated three sets of tiles, using the same coverage sizes and resolutions, with one exception. The highest resolution set has a resolution of 600 x 600 pixels for each tile. I was able to get the higher resolution for the first set, since the original source had a resolution of 21,600 x 10,800 pixels. Because of this, you can zoom into a graph further with Detailed Earth than with Simple Earth, before the quality of the background image begins to blur.

You can use transparency with both Simple Earth and Detailed Earth. Tip: To set the transparency, right-mouse-button over the graph and select the Customize... menu option. This will bring up the Customize dialog, where you can select the Background Map and assign a value for transparency. A valid value for transparency goes from 0.0 (completely transparent) to 1.0 (completely opaque).

In the graph below, you can see wind data plotted over a background map showing Lake Michigan. Originally, the arrows were difficult to see over the dark green of the map. After applying 50% transparency, the background map fades out, and the data becomes more visible.

wind data plotted over a background map showing Lake Michigan

Another nice feature of Simple Earth and Detailed Earth is the ability to wrap. The Earth is round, and when you cross 180° longitude, the Earth doesn't end. The longitudinal value continues from -180° and increases. The map will wrap continuously in the horizontal direction, much as the Earth does. The background map does not wrap in the vertical direction.

Simple Earth and Detailed Earth both support a geodesic scaling (thanks to Xan Gregg). In the images above, the Earth appears as a rectangle, where the width is twice as wide as the height. If we were to take this rectangle and roll it up, we would have a cylinder. This is known as a Mercator projection, named after the geographer and cartographer, Gerardus Mercator.

In reality, we know that the Earth doesn’t form a cylinder, but rather a sphere (or at least a close approximation to a sphere). So while the Mercator projection is useful for laying out the Earth with straight longitudinal and latitudinal lines, it distorts the actual size and shape of areas of the Earth. This distortion increases as we move from the equator toward the poles. To get rid of this distortion, you can use a geodesic scaling, which transforms the map to a more realistic representation of the Earth. To use the geodesic scaling, simply change the type of scale on the axes.

Tip: You can change the axes scale by selecting the right-mouse-button over the axis and then selecting Axis Settings…. Change the Scale to Geodesic or Geodesic US. Both choices will transform the map to a geodesic scaling. Use Geodesic US if you are viewing a map of the continental US and you want Alaska and Hawaii to be included in the map. It is important to note that you must set the scale to geodesic for both axes to get the transformation. You will not see a change in the map after setting only one of the axes.

In this graph, we see Simple Earth used as the background map with the axes set to use a geodesic scale. The axes lines are turned on as well. Notice the longitudinal lines are now curved, instead of straight. You might also notice how Greenland looks here compared to how it looked in the previous maps. (Greenland is the country at a high latitude in the Atlantic Ocean, predominantly covered in ice.)

Simple Earth used as the background map with the axes set to use a geodesic scale

Simple Earth and Detailed Earth are very easy to use, involving nothing more than selecting the option in the background map dialog. You don't need an Internet connection. The axes define the appropriate tiles for the background map.

But both Simple Earth and Detailed Earth do have their limitations. Tune in next time when you’ll hear somebody say, “Hey, when I zoom in to see my neighborhood, those Earth background maps get pretty fuzzy. What’s up with that?”

Got JMP 8? You Can Map, Too

 Background Maps, Data Visualization, JMP 8, JMP 9, Tips and Tricks  Got JMP 8? You Can Map, Too已关闭评论
10月 282010
 
By now, you are probably aware that JMP 9 delivers some cool new mapping features. I definitely encourage you to experience them firsthand. However, if you still use JMP 8, or have a colleague who has not upgraded to JMP 9 yet, a new script on the JMP.com file exchange might be just what you need.

The Polygon Script Creator (download requires a SAS.com login) allows you to choose one or more US states and generate a graphics script that you can add to any JMP graph. The graphics script lets you include a map background in your JMP graphs. The creator dialog has a resolution level choice of Low, Medium or High. Note that while High resolution will have the most detail, it can also take a long time to generate, especially if you have selected a lot (or all) of the states. For small numbers of states, High resolution should not take too long.

After you have generated the map that you want, there is a Create Script button that brings up the graphics script in a Script Editor window. This is the script that you can copy and paste into a JMP graphics customization window. (To get to a JMP graphics customization window, right-click on a JMP graph and select "Customize.” In the customization window, simply click the "plus" button to add a new graphics script and paste the generated script into the script box in the customization window.)

Here is an example of adding a map graphics script to a JMP graph, produced in JMP 8.0.2:
example of bubble plot with map of North Carolina