112017
 

People come from all over the world to attend this highlight of the season. It’s been a tradition for decades. Hotels book months in advance. Traffic is horrendous in the city center. The coveted tickets can cost thousands of dollars, but tens of thousands of people are lucky enough to score them. In […]

It's February. Game On! was published on SAS Voices.

112017
 

People come from all over the world to attend this highlight of the season. It’s been a tradition for decades. Hotels book months in advance. Traffic is horrendous in the city center. The coveted tickets can cost thousands of dollars, but tens of thousands of people are lucky enough to score them. In […]

It's February. Game On! was published on SAS Voices.

112017
 

In 2014, the federal government lost more than $125 billion to fraud, waste and abuse. And that’s just what we know about. While that number may sound incredible, those on the front lines of the government fraud fight know that it's all too real. The US government needs to change […]

U.S. government struggling with comprehensive fraud strategies was published on SAS Voices.

102017
 

Small matters matter. Imagine saving (or spending wisely) just 1 second of your time every hour. One measly second! During your lifespan you would save or spend wisely (1 sec-an-hour * 24 hours-a-day * 365 days-a-year x 100 years) / (3600 seconds-an-hour * 24 hours-a-day) = 10 days, a whole two week vacation!

While truncation vs rounding may seem to be insignificant in a given instance, the cumulative effect of either could be truly enormous, whether it’s truncation vs rounding of decimal numbers or of the SAS time values presented below.

From my prior post Truncating decimal numbers in SAS without rounding, we know that SAS formats such as w.d, DOLLARw.d, and COMMAw.d do not truncate decimal numbers, but rather round them.

However, SAS time value formats are somewhat different. Let’s take a look.

Suppose we have a SAS time value of '09:35:57't. As a reminder, a SAS time value is a value representing the number of seconds since midnight of the current day. SAS time values are between 0 and 86400.

TIMEw.d Format

Let’s apply the TIMEw.d format to our time value and see what it does.

If you run the following SAS code:

data _null_;
	t = '09:35:57't;
	put t= time5.;
	put t= time2.;
run;

you will get in the SAS log:

t=9:35
t=9

which means that this format does truncate both seconds and minutes. Conversely, if rounding were taking place we would have gotten:

t=9:36
t=10

HHMMw.d Format

Let’s run the same SAS code with HHMMw.d format:

data _null_;
	t = '09:35:57't;
	put t= hhmm5.;
	put t= hhmm2.;
run;

SAS log will show:

t=9:36
t=9

What does that mean? It means that HHMMw.d format rounds seconds (in case of truncating I would expect to get t=9:35), but truncates minutes (in case of rounding I would expect to get t=10, as 35 minutes are closer to 10 than to 9). A bit inconsistent, at least for our purposes.

Truncating SAS time values

This little research above shows that out of the two formats, TIMEw.d and HHMMw.d, it is perfectly fine to use the TIMEw.d format for the purpose of SAS time value truncation, for both minutes and seconds.

Regardless of the format used, you can also truncate your time value computationally, before applying a format, by subtracting from that value a remainder of division of that value by 60 (for seconds truncation) or by 3600 (for minutes truncation). For example, the following code:

data _null_;
	t = '09:35:57't;
	t_m = t - mod(t,60);
	t_h = t - mod(t,3600);
	put t= hhmm5.;
	put t_m= hhmm5.;
	put t_h= hhmm5.;
run;

produces the following SAS log:

t=9:36
t_m=9:35
t_h=9:00

Rounding SAS time values

Now that we’ve learned both the computational method and the TIMEw.d format method of truncation, how do we go about rounding? As long as the format behavior is consistent we can use its truncating functionality to convert it into the rounding functionality. In order to do that we just need to increase the original time value by 60 (seconds) for seconds rounding, and by 3600 (seconds) for minutes rounding. Truncation of that new value is equivalent to rounding of the original value.

Let’s run the following SAS code:

data _null_;
	t = '09:35:57't;
	t_m = t + 60;
	t_h = t + 3600;
	put t_m= time5.;
	put t_h= time2.;
run;

SAS log will show:

t_m=9:36
t_h=10

which means that our original time value '09:35:57't was rounded in both cases – seconds rounding and minutes rounding.

Now you know how to truncate and how to round SAS time values. And don’t forget about your lifetime 2-week vacation opportunity by saving a second every hour; or make it 2 seconds per hour and enjoy the full month off.

tags: SAS Professional Services, SAS Programmers, tips & techniques

Truncating vs rounding SAS time values was published on SAS Users.

102017
 

# -*- coding: utf-8 -*-
# @DATE : 2017/2/10 10:47
# @File : collection_usage.py

import collections

# counter 初始化
print(collections.Counter(["a", "b", "a", "b", "a", "c"]))
print(collections.Counter({"a": 2, "b": 3, "c": 1}))
print(collections.Counter(a=2, b=3, c=1))

c = collections.Counter()
print(c)
c.update("abababc")
print(c)
c.update({"a": 1, "d": 5})
print(c)

# counter access
c = collections.Counter("ababac")
for letter in "abcde":
print("{}: {}".format(letter, c[letter]))

# elements
c = collections.Counter("extremly")
print(c)
print(list(c.elements()))

# most common
c = collections.Counter()
with open("WordFilter.py", "r") as f:
for line in f:
c.update(line.strip().replace(" ", "").lower())

for letter, count in c.most_common(10):
print("{}: {}".format(letter, count))

# 数学运算
c1 = collections.Counter(["a", "b", "c", "a", "b", "b"])
c2 = collections.Counter("alphabet")

print(c1)
print(c2)
print(c1 + c2)
print(c1 - c2)
print(c1 & c2)
print(c1 | c2)


# ordered dict
d = {}
d["a"] = "A"
d["b"] = "B"
d["c"] = "C"
d["d"] = "D"
d["e"] = "E"
for k, v in d.items():
print("{}, {}".format(k, v))

print("Ordered Dict")
d = collections.OrderedDict()
d["a"] = "A"
d["b"] = "B"
d["c"] = "C"
d["d"] = "D"
d["e"] = "E"
for k, v in d.items():
print("{}, {}".format(k, v))




Counter({'a': 3, 'b': 2, 'c': 1})
Counter({'b': 3, 'a': 2, 'c': 1})
Counter({'b': 3, 'a': 2, 'c': 1})
Counter()
Counter({'a': 3, 'b': 3, 'c': 1})
Counter({'d': 5, 'a': 4, 'b': 3, 'c': 1})
a: 3
b: 2
c: 1
d: 0
e: 0
Counter({'e': 2, 'm': 1, 'l': 1, 'r': 1, 't': 1, 'y': 1, 'x': 1})
['e', 'e', 'm', 'l', 'r', 't', 'y', 'x']
e: 58
r: 56
t: 55
i: 48
o: 48
s: 48
d: 39
f: 36
n: 35
l: 33
Counter({'b': 3, 'a': 2, 'c': 1})
Counter({'a': 2, 'b': 1, 'e': 1, 'h': 1, 'l': 1, 'p': 1, 't': 1})
Counter({'a': 4, 'b': 4, 'c': 1, 'e': 1, 'h': 1, 'l': 1, 'p': 1, 't': 1})
Counter({'b': 2, 'c': 1})
Counter({'a': 2, 'b': 1})
Counter({'b': 3, 'a': 2, 'c': 1, 'e': 1, 'h': 1, 'l': 1, 'p': 1, 't': 1})
a, A
c, C
b, B
e, E
d, D
Ordered Dict
a, A
b, B
c, C
d, D
e, E

 
 Posted by at 9:13 下午
102017
 

深度学习在健康医疗领域应用

1. 深度学习

随着大数据和数据分析技术的不断发展,基于机器学习的数据驱动模型在健康医疗领域逐步得到推广和应用。深度学习作为在人工智能基础之上,是最近蓬勃发展的强大的机器学习工具之一,将会对未来人工智能产生颠覆性的作用。本文主要是对深度学习在健康医疗领域的应用进行综述,从而对目前的技术进行总结,对未来的应用进行展望,熟悉深度学习在健康医疗领域的应用。

2. deep learning Paper in health



3. 深度学习网络结构

深度学习算法在健康医疗中应用的统计 



3.1 Deep Neural Networks

常见的用于分类和回归的网络结构,隐藏层大于等于2;

3.2 卷积神经网络

适合二维数据,譬如图片,每一个卷积过滤将2d数据转化为3d输出,灵感来自于生物视觉;

3.3 循环神经网络

适合序列数据,输出依赖于前面的输出,共享参数;

3.4 Deep autoencoder

无监督学习,主要用于特征转换,输入节点数和输出节点数一致;

3.5 generative adversarial network

无监督学习,生成对抗神经网络,拟合出和原始数据分布一样的数据;

4. 深度学习计算包



5. 深度学习在健康医疗应用



6. 参考资料

Deep Learning for Health Informatics Daniele Rav`ı, Charence Wong, Fani Deligianni, Melissa Berthelot, Javier Andreu-Perez, Benny Lo, and Guang-Zhong Yang, Fellow, IEEE


 
102017
 

During my morning commute I heard an interesting news story about the merits and risks of the $100 bill. Apparently there are a lot of them in circulation, but no one knows exactly where they are. They are seldom used for legitimate business transactions because when a transaction reaches into the hundreds of dollars, the business parties tend to resort to a digital transaction (or maybe a good-old-fashioned personal check).

One stat from the story stuck with me: 80 percent of the currency value that is in circulation right now is in $100 bills. When I hear a number like that my first instinct is to find the data and verify it. It was easy to find on the US Federal Reserve: Currency in Circulation web site.

I could have copied and pasted the table for use in SAS, but the Fed already makes an ASCII version of the table available for nerds like me. With a bit of SAS code I was able to download the data, transpose it for analysis, calculate the currency values for with the counts of the various notes, and create a graph that verifies what I heard on the news. Indeed, much of our nation's cash wealth is floating around on a bunch of Benjamins.


The percentage of wealth represented in the $100 bill has grown over the past 20 years, perhaps at the expense of the far-more-ATM-friendly $20 bill. In 1996, the ratio of hundreds-to-twenties was 60/20. Today it's 79/12.

If we look at the percent breakdown by bill counts (instead of value), you can see the shifts a bit differently.

I created those two plots by calculating the percentages with PROC FREQ, and then using PROC SGPLOT to graph Year, using the percentages as the response value and stacking the data by Denomination. I used GROUPORDER=Data to keep the data colors and legend consistent across the different graphs. The raw values are interesting to examine as well, as it more clearly shows the trends for the past 20 years. Here are the two representations (cash value and bill counts) with their actual values and not just the percentages.

Raw values of currency in circulation

Raw counts of currency in circulation
An interesting aside: the $2 bill remains a novelty. I had the opportunity to spend a few of these recently, and the person receiving them had to be convinced that they were real money. It made me want to secure more $2 bills (a.k.a. "some Jeffersons") for future transactions -- that's the sort of trouble-making consumer that I like to be.

In case you're interested, here's the SAS program that I used -- try it yourself! (If using SAS University Edition, you'll have to download the text data first instead of relying on the FILENAME URL statement. Notes in the program.)

tags: SGPLOT

The post Visualized: US Currency in circulation, past and present appeared first on The SAS Dummy.

102017
 

Since the SAS 9.4 M2 release in December 2014, there have been several refinements and updates to the middle tier that are of interest to installers and administrators. In this blog, I’m going to summarize them for you. What I’m describing here is available in the newest SAS release (9.4 M4). I’ll describe them at a high level, and refer you to the documentation for details and how to implement some of these changes.

Security enhancements

Preserve your TLS Customizations:
For security purposes, many of you will manually add TLS configurations, either to the SAS Web Server, the SAS Web Application Server, or both. In addition, you may prefer to use your own reverse proxy server (such as IIS), either instead of, or in addition to, the SAS Web Server. Before the 9.4 M4 release, when upgrading or applying maintenance, you had to undo these custom configurations, perform the upgrade, and then apply the custom configurations again. Now, the upgrade will preserve them, making the process much easier. See Middle-Tier Security in the Middle Tier Administration Guide, Fourth Edition for full details.

Newer versions of OpenSSL are now provided (see doc for specific version numbers):
A Java upgrade enables enforcement of TLSv2. TLS is now considered the security standard for https connections, (SSL is obsolete) and this can be enforced with configurations to the SAS Web Server and the SAS Web Application Server. The new version of Java SAS is using (Ver 1.7+) now allows for this. One important thing to be aware of is that certificates are completely independent of which protocol you are using, and therefore any certificates you may have been using with SSL should work equally with newer TLS protocols.

Management of the trusted CA (Certificate Authority) bundle:
SAS now has a trusted CA bundle, that can be managed by the SAS Deployment Manager, in a new location:  SASHome/SASSecurityCertificateFramework/1.1/cacerts/. The CA certificates can be root certificates, intermediate certificates, or both. Here’s what the menu item looks like:

Middle Tier Changes and Upgrades in SAS 9.4 M4

Previously it was necessary to manually add your root/intermediate certificates to the Java truststore “cacerts,” located inside the JRE; now it’s done through the new interface. If you are on Windows, you must also add trusted CAs to the Windows store (as before), which will make them available to any browsers running there. This is documented at http://www.sqlservermart.com/HowTo/Windows_Import_Certificate.aspx and elsewhere online.

Security Support for SAS Web Applications – white list external sites, and HTTP request methods:
For added security, web sites hosting SAS web applications can now maintain a white list of external URLs that are allowed to connect in. This provides protection against Cross Site Request Forgeries, and other vulnerabilities. This is what the prompt looks like in the SDW:

Middle Tier Changes and Upgrades in SAS 9.4 M4

HTTP request methods can also be specified as allowed/not allowed. The list of URLs can be specified during installation in the SDW (shown above), or using the SAS Management Console. You can disable whitelist checking entirely, and you can add a “blacklist” or specific sites to always block. You can also block based on request method–ie, GET, POST, PUT, etc. See the Middle Tier Administration Guide for details.

Forward Proxy Configuration:
You can now set up SAS web applications to forward external URL requests through a proxy–here it’s called a forward proxy server. Many organizations do this behind their firewalls. See details for how to set this up in the administration guide.

Other miscellaneous changes:
As an administrator you can now force users to Log Off using SAS Web Administration Console.    You can also send emails to one or more users from the same window.  This is what the menu looks like:

Middle Tier Changes and Upgrades in SAS 9.4 M4

Faster start-up time for the SAS Web Application Server

JMS Broker (ActiveMQ) now uses Version 5.12.2 (fixed bugs).

SAS Web Server now uses version 5.5.2 and includes an updated mod_proxy_connect module for TLS tunneling.

References

SAS 9.4 Intelligence Platform: Middle Tier Administration Guide, Fourth Edition

Encryption in SAS 9.4, Sixth Edition

 

tags: SAS 9.4, SAS Administrators, SAS Professional Services, security

Middle Tier Changes and Upgrades in SAS 9.4 M4 was published on SAS Users.

092017
 

How important is reading to the skills gap? It's crucial. Through third grade, children are learning to read. After that, they read to learn. That is why reading proficiently by the end of third grade is one of the most reliable predictors of future success for children. Students who develop […]

Third-grade reading proficiency is key to reversing the “skills gap” was published on SAS Voices.