Does SAS still matter? Absolutely! And let me tell you why.

May 1 marked my 16th work anniversary at SAS and my 10th at SAS Middle East. Though I don't often mention my employer, this post is dedicated to the company I consider my second home and the primary source of my many lessons and experiences.

SAS - a brief window to the past

SAS Institute is a privately held software company located in Cary, North Carolina, founded in 1976 by Dr. Jim Goodnight and colleagues from North Carolina State University. SAS Institute initially focused on developing software for statistical analysis and data management. Its first product, SAS (Statistical Analysis System), quickly gained popularity among researchers and statisticians- especially in bio-research- establishing the company as a leading provider of statistical software in the 1980s. In the 1990s, SAS shifted its focus to commercial applications and expanded its offerings to include business intelligence software. The company introduced SAS Enterprise Miner in 1993, a data mining tool and SAS's flagship that helped businesses uncover patterns and relationships in their data.

Throughout the 2000s, SAS continued to expand its product offerings and established itself as a leader in the analytics industry. In 2000, SAS acquired DataFlux to expand its offerings in master data management, data quality, and data integration. In 2005, SAS released SAS9 - a major upgrade to its platform and product suite. The company acquired JMP, a data visualization and exploration tool, in 2007, and in 2011, it introduced SAS Visual Analytics, a platform for data visualization and exploration. In 2014, the company launched SAS Viya, a cloud-based platform for advanced analytics and machine learning.

My SAS - early years

My journey started as a Data integration / Business intelligence consultant, where my primary tasks revolved around data management. I passed my SAS Base programming certification and used SAS to read, transform, enrich, and design data for our customers' analysts to consume and generate insights.

SAS is unmatched when it comes to reading the data - at least from my experience!

1. Do you need to read data in diverse formats like CSV, TAB, Excel, text-based format, HTML, or XML?
2. Or from different sources like Oracle, MySQL, JDBC, ODBC, SAP, Teradata, and others?
3. Do you need to read text data in diverse encodings because you have offices across the globe?
4. Do you need to read proprietary text formats with multiple delimiters and with variable-length columns?
5. Do you need to read only particular pieces of information from huge text-based log files?
6. Do you need to read huge files which can't fit into memory? This is all just a piece of cake for SAS.

For SAS - this is the basics.

But maybe you want to read data from the internet - like reading an actual HTML file directly from a URL on the web server. Or maybe you need to dig a bit deeper and want to construct your own HTTP request. What if you want to read the URL, parse the data, extract only the email addresses, and store them in the final Excel file? What if you also want to read other links from the same file and crawl them further? Or maybe you want to collate the list of log files in a particular directory first, and then read and parse them one by one? All of this is again a no-brainer for SAS.

OK, I will stop here, as I hope I made it quite clear that SAS can probably extract and read any data you can practically think of. Also, please understand I have completely skipped the performance and optimization available when performing the above tasks using SAS.

My SAS - data management is fun, but customers want insights

Over time, I moved from data management tasks to broader use of SAS tools and capabilities, where data management was only one piece of the puzzle. In this phase, I used a suite of tools available to deliver data-driven solutions. These tools comprised:

1. Metadata management - a unified place to address a range of tasks like solution user management, roles, and access privileges, data source registration, scheduling, backup and auditing configuration, and many others

1. Data integration solution for managing the ETLs and data flows within the solution, batch processing and scheduling, data lineage, and others

1. Web Reporting and Dashboards to present and deliver the data, reports, or dashboards across the organization

Again, the above-described solutions and tools are only a fraction of what SAS had and provided to consultants and customers to fulfill their use cases.

Yet the above should provide a hint that - whether you wanted to consume and analyze the data as a resident data steward or you wanted to deploy an enterprise-wide data management platform that seamlessly transitioned into the web (thin-client) based distribution channel for diverse consumers and all of this supported by underlying IT and governance suite of tools - we all had it.

My SAS - the sky is the limit

Let me give you a practical example of using SAS tools to solve an actual POC use case in the fraud domain. Our task was to identify cycles (cyclic transactions) among corporate customers with intent to build good credit history, resulting in application fraud.

Rule limitation/parameters:

1. Amount of transfers between customers to be within +/- 10%
2. Applicable only to corporate customers’ transactions
3. Customers to be matched through name rather than actual ACCOUNT_NUMBER
4. Name to be matched using fuzzy matching technique
5. Customers to be matched against Blacklisted customers list
6. Algorithm depth level - L2 - L5
7. L1: A > A [me-to-me transactions were excluded as non-relevant]
8. L2: A > B > A
9. L3: A > B > C > A
10. L4: A > B > C > D > A
11. L5: A > B > C > D > E > A

The first part that could raise an eyebrow would be fuzzy matching of the names, but not for SAS, as we could use one of the many fuzzy matching techniques available within the platform:

1. Generalized distance
2. Levenshtein distance
3. Asymmetric spelling distance
4. Regular expressions
5. SAS Data Quality solution leveraging Match codes

In the final solution, we used SAS DQ Matchcodes primarily for easy accuracy adjustments.

The second tricky part was identifying the cycles. We identified more than one way of doing this - we tried and implemented three alternative solutions as we wanted to provide the one with the best performance. Three variants comprised of

1) Finding cycles using SAS Social Network Analysis tool

+ specifically designed for network analytics and application of network algorithms

- required post-processing as time constraints and amounts were not possible to filter within the algorithm

2) Finding cycles using SAS programming language (data step + macros)

+ custom-designed and built with all filtering conditions considered (applying conditions = Name, Amount filtering, etc.)

+ required very little storage space

- required a lot of CPU resources, and execution time grew exponentially

3) Finding cycles using SAS SQL [selected solution]

+ applied the filtering conditions and allowed easy manipulation and alteration of code

+ was by far the fastest of all algorithms

- required relatively high storage for execution

In the pictures above, you can see that the final solution was assessing approx. 1.5M transactions, among which numerous cycles have been identified. Although the report doesn't capture the algorithm's speed, execution for the above scenario took less than 1 min.

My SAS - the future

The above is just one example of SAS's capabilities, but thousands of other use cases can be solved in the same seamless way. The platform on which the SAS solution is built contains functions, procedures, and programming language constructs built and optimized through decades of development and fine-tuning.

So it is not surprising that "SAS AI and analytics platform is 30 times faster, more scalable and 86% more cost-effective than commercial and open source alternatives".

And though I don't want to boast, SAS is ranked as a leader by independent research companies like Gartner, Forrester, or Aite in categories like Data Science, AML, Fraud, Case Management, Data Integration, and Data Quality.

Looking ahead, SAS seems well prepared for what lies ahead, thanks to decades of market experience, accumulated assets and intellectual property, tools, and its platform.

Still, there is this one extra thing that matters the most, and here I will cite the founder of SAS - Jim Goodnight, who says: "I often say that 95% of my assets drive out of the front gate every night, and it’s my job to make sure they come back the next day."

SAS employees = my colleagues!

Knowing many amazing people in SAS globally is why I believe that SAS matters, and maybe even more than ever!

Thank you, Jim Goodnight, for SAS! Thank you - SAS - for 16 years of Curiosity!

-----------------------------------------------------------------------

If you reached here, you might feel something is still missing - what about analytics? Where are AI and machine learning?

Two points here:

1) In this blog, I chose to focus on the data part as it was, still is, and probably in the near future will be the most labor-intensive part of a data scientist job (you remember that 80-20 rule, right?)

2) Tapping into Analytics would further extend an already long blog post, so I will park it and address it in a separate article in the future