Jim Harris says data stewards are essential to analytics, providing life cycle management for data across the enterprise.

The post Data steward: The concierge of analytics appeared first on The Data Roundtable.

11月 272018

Jim Harris says data stewards are essential to analytics, providing life cycle management for data across the enterprise.

The post Data steward: The concierge of analytics appeared first on The Data Roundtable.

11月 272018

The universe of customer experiences, digital analytics, personalization and decisioning is massive. At times, it can seem as complicated and vast as the galaxy itself. With intricate subjects underneath this umbrella, you can lose direction, wander aimlessly, or feel a misleading sense of success or failure. When you lose vision, [...]

SAS Customer Intelligence 360: Model management for competitive differentiation [Part 1] was published on Customer Intelligence Blog.

11月 272018

Like many b2b and b2c organizations, our corporate website, www.sas.com, is a critical channel for how people learn about SAS and interact with us in the digital space. We have millions of visitors from around the world on a monthly basis looking to learn more about who we are, what [...]

Using SAS at SAS: The power of experimentation using SAS Customer Intelligence 360 was published on Customer Intelligence Blog.

11月 272018

Do you remember The Matrix movies, that started coming out in 1999? Hopefully this movie franchise didn't give you a fear of virtual reality and AI. The thing I remember most from the movie was the really cool slow-motion video effects (from multiple angles) in the fight scenes. And the [...]

The post Creating a 'Matrix-like' movie scroll animation, from a text file appeared first on SAS Learning Post.

11月 262018

Joseph Woodside discusses the evolution of digital transformation in healthcare in three eras.

The post Digital Disruption in Healthcare appeared first on SAS Learning Post.

11月 262018

Last week my colleague, Robert Allison, visualized data regarding immunization rates for kindergarten classes in North Carolina. One of his graphs was a scatter plot that displayed the proportion of unimmunized students versus the size of the class for 1,885 kindergarten classes in NC. This scatter plot is the basis for a statistical plot that is known as a funnel plot for proportions. The plot to the right shows a funnel plot that I created based on Robert's analysis.

The basic idea of a funnel plot is that small random samples have more variation than large random samples from the same population. If students are randomly chosen from a population in which some small proportion of children have an attribute, it might not be unusual if 40% of the students in a five-student class have the attribute (that's 2 out of 5) whereas it might be highly unusual to see such a high proportion in a 100-student class. The funnel plot enhances a scatter plot by adding curves that indicate a reasonable range of values for the proportion, given the size of a random sample.

For a discussion of funnel plots and how to create a funnel plot in SAS, see the article "Funnel plots for proportions." You can download the immunization data and the SAS program that I used to create the funnel plot for proportions.

- A scatter plot of the data. Each marker represents a school.
- A horizontal reference line. The line indicates the average proportion for the data. For these data, the statewide average proportion of unimmunized kindergarteners is 4.58%.
- A curve that indicates an upper confidence limit for the proportion of unimmunized students, assuming that the classes are a random sample from a population in which 4.58% of the students are unimmunized. When a marker (school) appears above the curve, the proportion of unimmunized students in that school is significantly higher than the statewide proportion.

This funnel plot uses the shape (and color) of a marker to indicate whether the school is public (circle), private (upward-pointing triangle), or a charter school (right-pointing triangle). The plot includes tool tips so that you can hover the mouse over a marker and see the name and county of the school.

The graph indicates that there are dozens of schools for which the proportion of unimmunized students far exceeds the state average. A graph like this enables school administrators and public health officials to identify schools that have a larger-than-expected proportion of unimmunized students. Identifying schools is the first step to developing initiatives that can improve the immunization rate in school-age children.

You can use a WHERE statement or BY-group processing in PROC SGPLOT to create a funnel plot for each county. A graph that shows only the schools in a particular district is more relevant for local school boards and administrators. The following graphs show the proportion of unimmunized students in Mecklenburg County (near Charlotte) and Wake County (near Raleigh), which are the two largest school districts in NC.

The first graph shows that Mecklenburg County has several schools that contain more than 60 kindergarten students and for which 25% or more of the students are unimmunized. In fact, some large schools have more than 40% unimmunized! In Wake County, fewer schools have extreme proportions, but there are still many schools for which the proportion of unimmunized students is quite large relative to the statewide average.

As Robert pointed out in his blog post, these are not official figures, so it is possible that some of the extreme proportions are caused by data entry errors rather than by hordes of unimmunized students.

The following graph shows a panel of plots for public, private, and charter schools. There are many public schools whose proportion of unimmunized students is well above the statewide average. For the private and charter schools, about 10 schools stand out in each group.

I think the plot of private schools is particularly interesting. When the media reports on outbreaks of childhood diseases in schools, there is often a mention of a "religious exemption," which is when a parent or guardian states that a child will not be immunized because of their religious beliefs. The report often mentions that private schools are often affiliated with a particular religion or church. I've therefore assumed that private schools have a larger proportion of unimmunized students because of the religious exemption. These data do not indicate which students are unimmunized because of a religious exemption, but the panel of funnel plots indicates that, overall, not many private schools have an abnormally large proportion of unimmunized students. In fact, the private schools show smaller deviations from the expected value than the public and charter schools.

In summary, I started with one of Robert Allison's graphs and augmented it to create a funnel plot for proportions. A funnel plot shows the average proportion and confidence limits for proportions (given the sample size). If the students in the schools were randomly sampled from a population where 4.58% of students are unimmunized, then few schools would be outside of the confidence curve. Of course, in reality, schools are not random samples. Many features of school performance—including unimmunized students—depend on local socioeconomic conditions. By taking into account the size of the classes, the funnel plot identifies schools that have an exceptionally large proportion of unimmunized students. A funnel plot can help school administrators and public health officials identify schools that might benefit from intervention programs and additional resources, or for which the data were incorrectly entered.

The post A funnel plot for immunization rates appeared first on The DO Loop.

11月 242018

First load necessary packages

Now, build our 2-D convolutional function that takes a custom filter matrix and comput the filtered output image matrix.

import pandas as pd

import numpy as np

from keras.models import Sequential

from keras.layers import Dense, Dropout, Conv2D

import keras.backend as K

import scipy, imageio

import matplotlib.pyplot as plt

from PIL import Image

%matplotlib inline

Then show original picture of my Jeep

# 首先将图片读入为矩阵

# 我们可以用pyplot的imshow()方法来展示图片

# 这是我曾经拥有的牧马人JK Rubicon Unlimited

#

img_data = imageio.imread('./pics/wranglerJK.jpg')

print(img_data.data.shape)

img = Image.fromarray(img_data, 'RGB')

plt.imshow(img)

添加图片说明 |

Now, build our 2-D convolutional function that takes a custom filter matrix and comput the filtered output image matrix.

`def my_init(shape, dtype=None):`

new_mat = np.zeros((shape[0], shape[1], 3, 3))

for i in range(shape[0]):

for j in range(shape[1]):

new_mat[:, :, i, j] = filter_mat

return np.array(new_mat, dtype=dtype)

def MyFilter(filter_mat):

print(len(filter_mat.shape))

if len(filter_mat.shape)!=2:

print('Invalid filter matrix. It must be 2-D')

return []

else:

kernel_size=filter_mat.shape

row, col, depth = img_data.shape

input_shape=img_data.shape

filter_size = row*col*depth

print(filter_size)

model = Sequential()

model.add(Conv2D(depth,

kernel_size=kernel_size,

input_shape=input_shape,

padding='same',

activation='linear',

data_format='channels_last',

kernel_initializer=my_init,

name='Conv')

)

model.add(Dense(1, activation='linear'))

model.compile(optimizer='sgd', loss='mse')

model.summary()

inX = model.input

outputs = [layer.output for layer in model.layers if layer.name=='Conv']

functions = [K.function([inX], [out]) for out in outputs]

layer_outs = [func([img_data.reshape(1, row, col, depth)]) for func in functions]

activationLayer = layer_outs[0][0]

temp = (activationLayer-np.min(activationLayer))

normalized_activationLayer = temp/np.max( np.max(temp))

return(normalized_activationLayer.reshape(row, col, depth))

Now, insert our own fixed filter matrix and get the output, using pyplot.imshow() to display the filtered picture. This time we throw in an edge detector.`filter_mat = np.array([-1, -2, -3, 0, 0, 0, 1, 2, 3]).reshape(3, 3)`

outLayer = MyFilter(filter_mat)

plt.imshow(outLayer)

Below is the filtered picture.11月 222018

This week I noticed that they've started building the lot where they sell Christmas trees near SAS (at the intersection of Maynard & Reedy Creek Rd). They put up a nice rustic wooden fence, and lights, and maybe even a fire pit to keep their workers warm. They sell some [...]

The post Finding a cut-your-own Christmas tree in North Carolina appeared first on SAS Learning Post.

11月 222018

This week I noticed that they've started building the lot where they sell Christmas trees near SAS (at the intersection of Maynard & Reedy Creek Rd). They put up a nice rustic wooden fence, and lights, and maybe even a fire pit to keep their workers warm. They sell some [...]

The post Finding a cut-your-own Christmas tree in North Carolina appeared first on SAS Learning Post.

11月 212018

Burger and fries, wine and cheese, peanut butter and jelly … some things just go better together. For organizations embarking on digital transformation, AI and IoT just go better together. These two distinct technologies; AI and IoT (or AIoT) are a natural fit. To take an analogy from the human [...]

AI and IoT, better together to accelerate digital transformation was published on SAS Voices by David Tareen