Data Visualization

2月 162011
 
If you followed my series of blog posts on background maps in JMP 9, you may have read the blog post titled, All About Background Maps in JMP 9: NASA Server and Web Map Service. In that blog post, I discussed two of the options for background maps that use a Web Mapping Service (WMS) to retrieve maps. One of the choices is a WMS server supported by NASA.

However, if you have JMP 9 and try to use the NASA option for background maps now, you will find that no map appears in the background. If you check the JMP log, you will see the message: "Unknown error while attempting to retrieve maps from WMS server." Recently NASA decided to stop supporting full WMS capabilities. So the NASA option for background maps in JMP 9 no longer functions.

But NASA has not bailed on us completely. The WMS server is not just another piece of space junk. It is still running and does support limited WMS functionality. The WMS functionality that it does support is more restrictive and requires the WMS requests to conform to a different standard than before. Unfortunately, it means having to make some code changes in JMP to make it function again. In fact, that is what I have done for the next maintenance update for JMP 9. So while the NASA option will no longer function properly in JMP 9.0 (or 9.0.1), it will return this spring in the 9.0.2 maintenance release. You will once again be able to produce background maps like the following.



These two maps show the impact site of a meteorite referred to as the Barringer Crater. It was named after Daniel Barringer, who was the first to suggest that it was created by the impact of a meteorite. The crater is located between Flagstaff and Winslow, Arizona. In the image above, the ridgeline of Diablo Canyon is visible to the west of the impact site, running in a north-south direction.



This situation does highlight one of the limitations of the WMS option for background maps. We (JMP) have no control over the WMS servers, and they can come and go as they please. You may have read another one of my blog posts in the series, titled All About Background Maps in JMP 9: WMS Explorer Add-In. That blog post describes a JMP add-in that I wrote called the WMS Explorer. The add-in helps you explore different WMS servers, and it comes with a pre-populated list of "known" WMS servers for you to explore. Some of the servers on that list have also stopped working since the release of the add-in.

Could it be that my blog posts are so popular and that so many people are using the add-in and hitting these WMS servers that the tidal wave of requests is overwhelming the servers and causing their demise? Sure, let's go with that! In any event, I think it is time for me to release another version of the add-in, with an updated list, including a few new servers that I have discovered. So don't forget to check the JMP File Exchange for this update coming soon, as well as many other new and updated JMP add-ins.

There are many WMS servers available on the Internet, and some of them have been very stable and continue to operate. So I hope you will continue to enjoy the WMS feature in JMP 9.
2月 152011
 
Tell us how in a paper or poster abstract for Discovery Summit 2011.

The Discovery Summit 2011 Steering Committee is inviting you and other JMP users to share how this statistical discovery software from SAS® has solved real problems. If your paper or poster is selected, you will present at the conference, Sept. 13-16 in Denver, the Mile High City.

When we asked last year’s Discovery Summit attendees what they found most valuable about the conference, many noted the breakout sessions featuring user presentations. They said they enjoyed:

• “Learning new release features, seeing them in action, watching problem-solving showcases presented by [the] finest analysts…”
• “The ability to see how other companies, industries and problem solvers are applying analytics and visualization in respect to specific business problems.”
• Hearing “new ideas for data analysis.”

In the words of attendees, “having lots of users mix together brought about great synergy.”

We believe that many of the greatest lessons learned at Discovery Summit 2011 will be passed from peer to peer. You have a unique perspective on the use of analytics in your field that, if shared, could leave fellow attendees better prepared to do their jobs and implement sound strategies.

Plus, there are perks.

Presenters will:

• Share the agenda with some of the finest, including our keynote speakers, who are thought leaders in statistics, technology and innovation.
• Receive complimentary conference admission.
• Enjoy top-notch accommodations from Sept. 13-16 if your paper is accepted.*
• Gather feedback from other attendees so you can further fine-tune your own analyses.
• Contribute to the conference conversation, advancing the use of analytics across forward-looking companies around the globe.
• Demonstrate to your colleagues and managers how higher analytics benefit your organization.

We hope you’ll participate. Learn more and submit a paper or poster abstract by March 18, 2011, at 5 p.m. ET for consideration.

*Not applicable to poster presenters.
2月 052011
 
I promised the 400 people who attended Chuck Pirrello's Feb. 3 live webcast on using the JMP Add-In for Microsoft Excel a link to a JMP and Excel demo he recorded for you.

Did you miss the live event? No problem. You can catch the 10-minute demo at your leisure.

Want to try some of the data manipulation Chuck demonstrated? Just download his Sales by Product Name files from the JMP File Exchange.

The interactive Q&A was one of the most valuable aspects of Chuck's live webcast. I spoke with Chuck and with Paul Nelson, our JMP Add-In for Microsoft Excel developer, to get answers to some of the common questions that arose yesterday.

Q: What version of JMP do I need to run the JMP Add-In for Excel?
A: JMP 9 or JMP Pro.

Q: What platforms work with the JMP Add-In for Excel?
A: The add-in currently runs on Windows only.

Q: With what versions of Microsoft Excel is the JMP Add-In for Excel compatible?
A: 32-bit versions of Microsoft Office 2003 or later. The Add-In for Excel does not work with 64-bit versions of Microsoft Office/Excel. Look for 64-bit Office compatibility in a JMP 9 update later this year.

Q: If I have JMP 8, how can I look at the Prediction Profiler that Chuck demonstrated?
A: Chuck's demo (and data) on Predicting the Impact of Business Decisions will walk you through the steps using JMP 8.

Live Mastering JMP webcasts on other JMP 9 and JMP Pro topics -- such as predictive modeling, interactive data mining and mixture designs -- continue through May. Consider registering for one or more topics that interest you.
1月 102011
 
Bubble plots in JMP are a great way to see how values change over time. But they also allow for the viewer to see so many variables (or dimensions) at one glance. So they can be a valuable visualization tool to view data without a time dimension. Think of that dimension as a page dimension for a book of reports -- or in this case, a set of graphs.

In the example below, which uses performance results by product, I selected the Distribution Channel in place of my time dimension. I created four quadrants and can see the other variables as shown below:



Now, when I click on the Step button, I can page through all my channels and see the results by channel. You'll see three "pages" of bubble plots.







So the bubble plot is not only a way to show changes in values over time. You can use it in very creative ways to show data over any variable.

Can you think of other examples where you could use the bubble plot in this way?
12月 202010
 
This blog post is the final one in a series about image-based background maps in JMP 9.


Part I of the series introduced background maps, which are a new feature in JMP. It explained why background maps are useful and showed how to access a background map through the user interface.

Part II of the series discussed Simple Earth and Detailed Earth. These options are the background maps that are installed as part of the JMP product. Because they are local to the machine, they don't require any Internet connection or rely on server availability. But they have a limited resolution, and the quality of the imagery starts to degrade when you zoom in too close.

Part III of the series discussed the Web-based options for background maps: NASA and Web Map Services. These options support the WMS api and let you retrieve maps from a WMS server on the Internet. A WMS server can often generate a map at a higher resolution, allowing you to zoom in on the graph.

Part IV of the series discussed a JMP add-in called the WMS Explorer. This add-in helps to find WMS servers that are available online. It also gives a preview of what locations the WMS server covers and what types of maps it can return.



I invite you to read these blog posts if you haven't already. I hope they will answer any questions you have about the image-based background maps in JMP 9. I'd also like to point out that others at JMP have written blog posts about the vector-based background maps.

One thing I haven't discussed yet is how to access the background map functionality through JSL. You can find an example of this in the Samples Data directory. Run JMP and open the sample data file called Hurricanes.jmp. Edit the script called Bubble Plot with Map. Look for the command in the script that begins with Background Map.

To figure out the correct parameters for the background map command, simply look at the background map dialog. The wording on the dialog is the same as the JSL commands. To create a background map, simply use the command Background Map().

There are two types of background maps you can specify: Images() and Boundaries(). Each of these then takes a parameter, which is the name of the map to use. The name is one of the maps listed in the dialog. For Images(), the choices are "Simple Earth," "Detailed Earth," "NASA" or "Web Map Service." If you elect to use "Web Map Service," then there are two additional parameters: the WMS URL and the layer supported by the WMS server. For Boundaries(), the choices vary since boundaries can be user-defined. But a typical choice might be "World Countries."

So if I wanted to add a background map that used the "Detailed Earth" imagery and the "World Countries" as a boundary, the command would look like this:

  Background Map ( Images ( "Detailed Earth" ), Boundaries ( "World Countries" ) )

If I wanted to change that to use a WMS server, then the command might look like this:

  Background Map (
     Images ( "Web Map Service", "http://wms.jpl.nasa.gov/wms.cgi", "BMNG" ),
     Boundaries ( "World Countries" )
  )

Note the spacing doesn't matter. I just did it for readability. And here is what the graph would look like:



Tip: Of course, you can always create your graph and add a background map through the user interface. Then from the little red triangle (lrt) use the Script->Save Script to Script Window menu item to see the JSL that gets generated. I find this technique very helpful for learning JSL in general.

So what else can you do with background maps in JMP? Well, apparently you can use them to create the company holiday card. Check it out! Whatever holiday you choose to celebrate at this time of year, I'd like to wish you a happy one. And I hope you have a happy, healthy and successful 2011.

Peace!
12月 132010
 
Graph Builder strives to help you see your data without adding any interpretation. That's why the default view shows the actual data points and the summary element is a spline smoother instead of a regression line. However, even these elements bring some baggage. Data markers have over-striking issues when many are present, and a million of them can take several seconds to draw. The smoother shows an average trend, whether it's representative or not.

Looking at last year's power data from the SAS Solar Farm, the default Graph Builder view is a smoother. Since there are 50,000 data points, the Marker element is turned off by default.



The curve looks reasonable, but without the markers we can't tell if there's more to the story. Turning on markers, we get:



This illustrates the over-striking issue and doesn't tell us anything unless we notice that the scale has changed. One way to help with over-striking is to set the transparency of the markers to something like 0.1.



That's a big improvement, though the bottom markers seem maxed out and hard to compare. Plus, transparency doesn't solve the speed issue. For 50,000 points, JMP is still responsive. But at 500,000 points, it would start trying your patience.

A new alternative for overcoming these issues is the Contour element, which, in the absence of a color variable, will show shaded contours representing point density.



If you squint your eyes, it looks a lot like the view of markers with transparency, but it doesn't have the over-striking or performance issues. With either view, you get the sense that the smoother doesn't go through the densest part of the data, which suggests there are other factors involved. In this case, we might conclude that there are some sunny days and some cloudy days but not many in the middle.

I created a dummy variable called "Sunny" and assigned it to the Overlay role just to illustrate that the density contours can be overlaid.



That doesn't tell the whole story, but we can tell we're getting closer. (We know from previous analysis that time of year makes a difference, too.)

The density contour may look strange if you're not expecting it, which is why it's turned off by default. However, there are new preferences for Graph Builder in JMP 9 in which you can not only control the default elements but also the cut-offs for when Graph Builder switches to the summary view.



For instance, you can set the Continuous Alternate to Contour if you want to see the density contour whenever the Points Limit is exceeded.
12月 012010
 
7,000 -- that's the big number everyone was talking about last week in Paris. 7,000 what?

7,000 Euro/m², that’s the average price prospective buyers now have to pay for properties in Paris. If you are not used to m², 1 m² is roughly equal to 10.8 ft², and whichever way you look at it, that’s expensive.

However, this number is an average. As Kaiser Fung, a speaker at the Discovery Summit earlier this year, wrote in his book Numbers Rule Your World: "Averaging stamps out diversity, reducing anything to its simplest terms. In doing so, we run the risk of oversimplifying, of forgetting the variations around the average.”

Paris is divided in 20 districts, known as “arrondissements.” These are laid out in a spiral or snail shape from the 1st in the center to the 20th, a bit further out, reflecting how the city developed over the centuries. Using the new mapping capabilities of JMP 9, I could very quickly create a map of Paris and use it to go beyond the newspaper headline.

In a recent post, Justin Mosiman explained how to use the Custom Map Creator, a JMP add-in, to build your own map. Just a few minutes after downloading Justin's add-in, I was able to turn data into information and could see beyond the average.

What is quickly apparent is that property prices vary from arrondissement to arrondissement. Not only can we see that the center is more expensive than the edge (not that surprising), but we also see that the east tends to be more affordable than the west. In addition, the 6th district (near Notre Dame Cathedral) is the most expensive district.



So are property prices going through the roof (literally) everywhere? Well, actually, in most neighborhoods the answer is yes since prices went up about 4 percent in only three months. Looking at both maps, you see that the most expensive district (the 6th) is also where prices are rising the fastest. On the other hand, Paris’ most central area (the 1st), has seen a 1.8 percent quarterly drop.



Anyway, let’s just hope that the big number -- 7,000 -- does not get any bigger!
11月 302010
 
Many of you are already aware of the new powerful mapping capabilities in JMP 9 that enable you to visualize geographic-based data without having to provide any longitude/latitude or shape information. For example, I can open a data table in JMP that has two columns – the names (or abbreviations) of US states in one column, crime rate in another – and Graph Builder will draw a map of the US for me coloring the states by the value of crime rate.




This is possible because JMP 9 automatically comes with the shape (.shp) and database (.dbf) files for US states as well as many other geographic shapes such as countries, US counties and provinces of various European countries.

As a JMP Systems Engineer, I often demonstrate this new capability using the crime rate example above. Folks get excited and usually ask which other shapes they can map out. I am happy to tell them about the others that are provided and that they can also add their own shape files very easily to map out whatever shapes they want.

Unexpectedly, I had to prove this claim on a customer phone call recently! This customer was interested in the same mapping capability for US Metropolitan Statistical Areas (MSAs). As I was explaining how easy it would be to find the .shp/.dbf files and add them to the appropriate directory, one of the customers on the phone call found the correct files on the US Census website.

“Hey, Mike, I think I have the files right now, so why don’t you give it a shot?” the enthusiastic customer said. I think we were all amazed, myself included, at how simple this really was:

• I opened the .shp and .dbf files obtained from the Census website in JMP.
• I verified that Shape ID was the first column in the shp file and that it contained X and Y columns (which represent the longitude and latitude coordinates).
• I saved this .shp file to the Maps folder as “US-MSA-XY.jmp.”
• I created a Shape ID column in the .dbf file, which is a column that simply mimics the row number. A nice trick to do this easily: Visit Column Info in a new column and choose Initialize Data>Sequence Data. I moved this column to be the first one in the table.
• Then I assigned the Map Role column property to the name of the MSAs. This is how JMP knows to draw a map when the user provides the name. I right-clicked the CBSA column and chose Column Properties>Map Role and then chose Shape Name Definition from the ensuing drop-down box.
• I saved this dbf file to the Maps folder as “US-MSA-Name.jmp”

That’s it! I reopened the “Name” file I had just saved and dragged CBSA into the shape zone in Graph Builder to verify that the shape name was correctly identified by JMP. So, not 10 minutes after I mentioned it as a possibility, we were viewing an MSA-level map in Graph Builder. After much applause (in my head), these customers were visualizing their MSA data.




So Google “esri shape” and the name of the shape you are looking for, and start adding your own! And keep your eyes open for another blog post about using this method to analyze Afghanistan Province-level data.

The JMP directory that contains these files can be found at the following paths:

Windows:
C:\Program Files\SAS\JMP\9\Maps

Mac:
/Library/Application Support/JMP/9/Maps
11月 222010
 
In my previous blog post, I talked about using a Web Map Service (WMS) to generate a background map. The WMS feature is more flexible than the static background maps provided as part of JMP. But it requires the user to find an appropriate WMS server and then determine specific information about the server, such as layer names.

This can be a time-consuming task in itself. Searching the Web will help you find WMS servers that are publicly available. But it can be more difficult to figure out what the layer names are and what the resultant map will look like. To help with this discovery process, I have taken advantage of another JMP 9 feature called the add-in. An add-in is a way to extend the functionality of JMP. Add-ins (including this one) can be downloaded from the add-ins page on the JMP File Exchange.

I created an add-in called the WMS Explorer. The WMS Explorer add-in is designed to make it easier for you to find a WMS server, query the server for layer names and show you what a map will look like. Here is what the WMS Explorer add-in looks like.

WMS Explorer add-in

At the top of the window is a text field where you can type in a URL to a WMS server. You can find possible WMS servers by searching the Web. After you enter a URL, either hit the enter key or click on the Get Layers button. This will send a query to the WMS server and return a list of layers that the server supports. If you do not know the URL for a server, you can select one from the drop-down list, directly below the text field. Selecting a URL from the list will populate the text field and submit the query for layer names.

The graph, below the drop-down list, will now contain a boundary map. Initially, the graph will show the Earth, with each country outlined. To the left of the graph is a list of layer names that the server supports. Select a layer name, and another request will be made to the server. This request will return a map that will be set as the background map on the graph. Here is an example where I selected the first URL in the drop-down list, the NASA server, and chose BMNG as the layer name.

Example using NASA server and BMNG as layer name

You can select the different layer names to see what each map will look like.

The graph is a typical graph in JMP, which means that all the regular JMP controls are available to you. You can adjust the axes or use the zoom tool (found on the hidden menu bar) just as you would in JMP. You can also right-mouse-button (RMB) to select Size/Scale->Size to Isometric to get your graph back into a proper aspect ratio. RMB will also allow you to select Background Map, where you can adjust the boundary map. Here is an example where I entered a URL for a WMS server in Spain and zoomed in on the Strait of Gibraltar. In this map, Spain is to the north and Morocco is to the south.

Example using WMS server in Spain, creating map zoomed in on Strait of Gibralter

Occasionally, a request to a WMS server will result in an error. At the bottom of the add-in window, there is an area for any error messages. You may try a server that no longer exists or is temporarily down. Or you may select a layer name that the server no longer supports. In a case like this, instead of returning a map, the server may return an error message that will be displayed in this area. Any error message that is displayed is a message that came directly from the WMS server.

Once you find a map that you want to use, simply note the URL in the text field and the name of the selected layer. This is the information you will need to use the WMS background map feature in JMP.

I hope you enjoy the WMS Explorer add-in and find it useful. If you find a good WMS server, please send it to me. I’ll try to keep an up-to-date list available to share with other JMP users. As always, feel free to share your comments and suggestions.
11月 192010
 
I started to work at JMP this past summer as a student intern. One of my first projects was to create a background map for the office temperatures study. As you probably know, JMP 9 can graph map shapes using built-in map files. JMP can also work from user-created shapes, which can be imported from ERSI SHP files or created from scratch. To make the latter process easier, I created the visual Custom Map Creator add-in.

Shapes in JMP are defined by two different data tables that work together, a name file and an XY file. The name file contains the ID and name of the shape that you are defining. The XY file contains four different columns that define the shape coordinates. The first column is the ID of the shape that is defined in the name file. There is also a part ID, which is used if you have multiple parts to a single shape. For example, if you were defining Hawaii, each island would be a part, and all of the islands would have the same shape ID. Finally, the X and Y columns specify the location of one particular point in the boundary.

In the image below, shape 50 has six rows that correspond to the six points. Since the shape is not broken into multiple pieces, it has one part.

a shape that has 6 rows and 6 points

The process of creating the shapes by hand is tedious at best, but the Custom Shape Creator add-in allows you to create the different shape boundaries with the click of a button. All you have to do is drag the image that you want to model into the Custom Map Creator window and use your mouse to define each boundary.

The image shown below is the start of my recreation of the offices map. The Custom Map Creator has a few different visualization options to help you make your boundaries. The line width and color can be changed, in addition to being able to fill the shapes in with a random color (as enabled in the picture below).



I recreated the office map in about 25 minutes with this add-in. No more manually entering in coordinates! As an added bonus, the magnifying glass still works so your map can be really detailed.



The Custom Map Creator is available on the JMP File Exchange. If you have any awesome maps that you created and would like to share I’d love to see them in the comments.