3月 282017
 

Did you know that PROC SQL captures the record count for a result set in a special automatic macro variable? When you create a subset of data to include in a report, it's a nice touch to add a record count and other summaries as an eye-catcher to the report title. I often see the following pattern in SAS programs, which adds an extra step to get a record count:

proc sql noprint;
 
 create table result 
  as select * from sashelp.cars
  where origin='Asia';
 
 /* count the records in the result */
 select count(model) into :resultcount
  from result;
quit;
 
title "Summary of Cars from Asia: &resultcount. records";
proc means data=result;
run;

This creates a report with an informative title like this:

Here's the tip. Instead of including a SELECT INTO step that's going to make another pass through the data, you can rely on the &SQLOBS automatic macro variable. This variable holds the record "result set" count from the most recent SELECT clause.

proc sql noprint;
 create table result 
  as select * from sashelp.cars
  where origin='Asia';
 
 %let resultcount=&sqlobs;
quit;
 
title "Summary of Cars from Asia: &resultcount. records";
proc means data=result;
run;

Because SAS replaces the value with each subsequent SELECT clause, it's important to assign it to another macro variable immediately if you intend to use it later. Here's the result:

Not only is this more efficient, but SAS automatically trims the whitespace from the SQLOBS variable so that it looks better in a TITLE statement. If you're using SELECT INTO to populate macro variables for other reasons, you can use the TRIMMED keyword to achieve the same effect.

proc sql noprint;
 
 create table result 
  as select * from sashelp.cars
  where origin='Asia';
 
 %let resultcount=&sqlobs;
 
 select avg(mpg_highway) into: AvgMpg TRIMMED
  from result;
 
quit;
 
title "Summary of Cars from Asia: &resultcount. records, &AvgMpg. MPG Average";
proc means data=result;
run;

See also

The post How many records are in that PROC SQL result? appeared first on The SAS Dummy.

3月 272017
 

Here in the US, it's the nationwide men's college basketball tournament season! Therefore let's use some data from the previous years' tournaments to sharpen our analytics & visualization skills... But before we get started, I must mention (brag?) that my alma mater, NC State University, won this tournament in 1983. [...]

The post Let's analyze 32 years of basketball tournament data! appeared first on SAS Learning Post.

3月 272017
 

Here in the US, it's the nationwide men's college basketball tournament season! Therefore let's use some data from the previous years' tournaments to sharpen our analytics & visualization skills... But before we get started, I must mention (brag?) that my alma mater, NC State University, won this tournament in 1983. [...]

The post Let's analyze 32 years of basketball tournament data! appeared first on SAS Learning Post.

3月 272017
 

When senior leaders at the University of Louisville (UofL) approached Vice Provost Bob Goldstein in early April 2016 with a request for a fully functioning data visualization platform by start of the 2016 fall semester—just four months away—he did not panic. Instead, Goldstein, along with Becky Patterson, Executive Director of [...]

Data visualization helps University of Louisville achieve its 2020 strategic plan was published on SAS Voices by Georgia Mariani

3月 272017
 

Did you know that you can check a SAS macro variable to see if ODS graphics is enabled? The other day I wanted to write a SAS program that creates a graph only if ODS graphics is enabled. The solution is to check the SYSODSGRAPHICS macro variable, which is automatically updated by the SAS system whenever ODS graphics is enabled or disabled. (Thanks to Warren Kuhfeld for this tip!) The SYSODSGRAPHICS variable has the value 0 for "off" and 1 for "on."

For example, the following SAS/IML program checks to see if ODS graphics is on. If so, it creates a bar chart of a categorical variable. If not, it computes the count for each category and prints a frequency table:

ods graphics on;         /* -OR-  ods graphics off */
proc iml;
use sashelp.cars;  read all var "Origin";  close;
 
if &SYSODSGRAPHICS then  /* is ODS graphics enabled? */
   call bar(Origin);     /* optionally display a graphical summary */
/* always display a tabular summary */
call tabulate(Categories, Counts, Origin);
print Counts[colname=Categories];

The SYSODSGRAPHICS automatic macro variable is a recent addition to the automatic macro variables in SAS. You can read Rick Langston's 2015 paper to discover other (relatively) new macro features in SAS 9.3 and SAS 9.4.

SAS has many other automatic macro variables. In general, you can use automatic macro variables in SAS to check the status of the SAS system at run time. Some of my other favorite automatic macro variables (which some people call system macro variables) are the following:

  • SYSERR, SYSERRORTEXT, and SYSWARNINGTEXT: Provide information about errors and warnings from SAS procedures. In my book Statistical Programming with SAS/IML I show how to use these macro variables in conjunction with the SUBMIT/ENDSUBMIT statements in SAS/IML.
  • SYSVER and SYSVLONG: Provide information about your version of SAS. You can use these macro variables to execute newer, more efficient, code if the user has a recent version of SAS.
  • SYSSCP and SYSSCPL: Provide information about the operating system. For example, is SAS running on Windows or Linux?

Do you have a favorite automatic variable? Leave a comment and tell me how you use automatic variables in your SAS programs.

The post Is ODS graphics enabled? Use automatic macro variables to determine the state of SAS appeared first on The DO Loop.

3月 262017
 
文章主要是关于用户意图预测和pinterest视觉搜索引擎相关的探索

精华

paper:Predicting Intent Using Activity Logs: How Goal Specificity and Temporal Range Affect User Behavior

We find that goal specificity is bimodal – users tend to be either strongly goal-specific or goal- nonspecific. 

Goal-specific users search more and consume less content in greater detail than goal-nonspecific users: 

they spend more time using Pinterest, but are less likely to return in the near future. 

Users with short-term goals are also more focused and more likely to refer to past saved content than users with long-term goals, but less likely to save content for the future. 

Further, intent can vary by demographic, and with the topic of interest. Last, we show that user’s intent and activity are intimately related by building a model that can predict a user’s intent for using Pinterest after observing their activity for only two minutes.

paper:Understanding Behaviors that Lead to Purchasing: A Case Study of Pinterest

We analyze the purchasing behavior of nearly three million Pinterest users to determine short-term and long-term signals in user behavior that indicate higher purchase intent. We find that users with long-term purchas- ing intent tend to save and clickthrough on more content. However, as users approach the time of purchase their ac- tivity becomes more topically focused and actions shift from saves to searches.

We further find that purchase signals in online behavior can exist weeks before a purchase is made and can also be traced across different purchase categories.

paper:Visual Discovery at Pinterest

pinterest 视觉发现引擎,经验总结:

(1)advances in computer vision, especially the use of convolutional networks and GPU acceleration, have led to significant improvements in tasks such as image classifica- tion and object detection.

(2)a substantial number of users prefer using discovery systems to browse rather than to search.

实验

(1) FEATURE REPRESENTATION 特征表示学习

基于已有的模型,adopted and evaluated several popular classification models such as AlexNet[20], GoogLeNet [35], VGG16 [32], and variants ResNet101 and ResNet152[12].

(2) OBJECT DETECTION

Extracting objects from images not only allows us to build novel discovery experiences (e.g. Section 8), but also improves user engagement metrics;算法Faster R-CNN

(3)PINTEREST RELATED PINS

相关性推荐Related Pins is a pin recommendation system that lever- ages the vast amount of human-curated content on Pinter- est to provide personalized recommendations of pins based on a given query pin.Covnet features for recommendations;




 
3月 252017
 

python 获取tensorflow课程讲义

# -*- coding: utf-8 -*-
# @DATE    : 2017/3/25 11:08
# @Author  : 
# @File    : pdf_download.py

import os
import shutil
import requests
from bs4 import BeautifulSoup
import urllib2

def download_file(url, file_folder):
    file_name = url.split("/")[-1]
    file_path = os.path.join(file_folder, file_name)
    r = requests.get(url=url, stream=True)
    with open(file_path, "wb") as f:
        for chunk in r.iter_content(chunk_size=1024 * 1024):
            if chunk:
                f.write(chunk)
    r.close()
    return file_path

def get_pdfs(url, root_url, file_folder):
    html = urllib2.urlopen(url)
    soup = BeautifulSoup(html, "lxml")
    cnt = 0
    for link in soup.find_all("a"):
        file_url = link.get("href")
        if file_url.endswith(".pdf"):
            file_name = download_file(file_url, file_folder)
            print("downloading {} -> {}".format(file_url, file_name))
            cnt += 1
    print("downloaded {} pdfs".format(cnt))

def main():
    root_url = "http://web.stanford.edu/class/cs20si/lectures/"
    course_url = "http://web.stanford.edu/class/cs20si/syllabus.html"
    file_folder = "./course_note"
    if os.path.exists(file_folder):
        shutil.rmtree(file_folder)
    os.mkdir(file_folder)
    get_pdfs(course_url, root_url, file_folder)

if __name__ == "__main__":
    main()

 
 Posted by at 7:42 下午
3月 242017
 

In a recent presentation, Jill Dyche, VP of SAS Best Practices gave two great quotes: "Map strategy to data" and "strategy drives analytics drives data." In other words, don't wait for your data to be perfect before you invest in analytics. Don't get me wrong -- I fully understand and [...]

Don't let data issues delay analytics was published on SAS Voices by David Pope

3月 232017
 

SAS® users have an easy and convenient way to quickly obtain useful information (referred to as metadata) about their SAS session with a number of read-only SAS DICTIONARY tables or SASHELP views. At any time during a SAS session, information about currently defined system options, libnames, tables, columns and their [...]

The post Exploring the content of the DICTIONARIES table and VSVIEW SASHELP view appeared first on SAS Learning Post.

3月 232017
 

If you're a fan of SAS' ODS Graphics, you probably know that it does pretty much everything except geographical maps. But it's flexible enough that you can "fake it 'till you make it"! This example describes how to fake a geographical (choropleth) heat map using Proc SGplot polygons. In my [...]

The post Bringing the heat! - Creating heat maps with proc sgplot ... appeared first on SAS Learning Post.