Jim Goodnight | Intelligence apps look to future

It's no surprise that SAS Institute Inc., the world's largest privately held software company, makes its home in Cary, N.C. With headquarters so near three of the state's top universities'Duke University, North Carolina State University in Raleigh and the University of North Carolina at Chapel Hill'the company has its pick of math and computer science graduates. Such a wellspring of talent helps the company stay ahead of the game in analytic software, which the company has offered since 1976.Unlike many CEOs, SAS headman Jim Goodnight has programming experience. He has actually done a considerable amount of coding for the company and occasionally still leads projects. He also knows his stats. He holds both bachelor's and master's degrees as well as a doctorate in statistics from N.C. State. Before working at SAS, he was an N.C. State faculty member, and he continues to serve as an adjunct professor.GCN associate writer Joab Jackson interviewed Goodnight at SAS' annual Government Executive Event in Washington.Goodnight: It certainly has been a trend at some cutting-edge organizations. In the past, enterprise resource planning systems basically gave you information about what has gone on before. They didn't really try to help you forecast what will happen in the future. Now we're seeing a trend of, say, retailers forecasting demand for goods so they can ... optimize prices to maximize profit. We have a lot of work going on in optimization.Goodnight: We've done quite a lot in making the SAS software multithreaded. If you have a single-threaded software, it doesn't matter how many processor cores you have, the application will only use one of them. For our 9.1 release, we threaded a number of the data-mining procedures that operate on large amounts of data. We will continue over the next few years to multithread everything we have. Some of our core summarization routines and sorting routines can quickly be threaded.[A software developer] has to figure out where to split out different processes to run simultaneously. It really can't [be] done effectively by just breaking the problem up into a lot of little pieces and trying to sew them back at the end. Most of the algorithms we deal with have places where they can be broken up. We can give the first processor the first 100,000 records and the second the next 100,000. [On an eight-core processor], we can read eight streams into eight different threads at once. If we can achieve that, we can really get the speed going.Goodnight: I'm trying to move away from having to support too many application servers. You need them for persistence. Other than that, we don't use many of their features. We support WebLogic and Websphere and bundle the open-source TomCat for no charge.About three years ago, we thought that every customer would end up using one standard app server, and we would support that standard. Well, it hasn't turned out that way. App servers are tied in with the individual applications. Instead of there being just one app server, every vendor has to ship an app server with their application. Oracle Corp. has its own app server, Sun Microsystems Inc. has its own, SAP has its own, and so on. There are a dozen different app servers and no one is standardizing on any of them.My goal is to slowly get away from this. We support 12 different operating systems. Now if I also have to test four different app servers for those 12, the combinations just get unbelievable.Goodnight: They are basically offer[ing] query and reporting tools. I have noticed that they have added a few analytical pieces, regression analysis and things like that. But you have to realize we've been building these things for 30 years. We have nearly every analytical process known to man in our software.Goodnight: Things like neural networks. These are very complex models that supposedly mimic the neural network in your brain. I don't particularly adhere to that belief. It is just a model, but there [are] a lot of parameters that have to be estimated, and you estimate those by reading through the data and building various matrices to do the estimations.We've had the general linear model since 1976. If you have, say, a person's age, weight and height and want [to] forecast how fast they could run, then you'd use a linear model with those three variables to predict how fast they could run.In addition to this procedure, we have about 200 other models. We have a logistic regression model that, say, a telephone company could use [to] gauge the probability [that a given customer will] stop using its service.Say you have 15 million customers, we can look at the data on those customers'how many minutes they used a month, how much they were charged, how quickly they paid their bills'and [divide up] those who have left and those who are still there. You would hope that the data collected would have enough information to predict the current customers who are most likely to quit. If you can [find a way to] keep those people, you can save a bundle of money not having to go after replacements.Goodnight: We've shipped about 20,000 copies out to our customer base of 42,000. Quite often, people are so satisfied with the current release they don't want to upgrade, so it usually takes a couple of years for people to convert. And some people refuse to convert ever. We have a number of organizations still running SAS 6 because they run just one application on it, and they just want to keep running on it.We continue to improve SAS Version 9. We have two additional service packs for this year that are some minor upgrades and improvements to the basic package. Anytime we come up with performance area problems, we'll work on those. Unlike most other software companies, we try to get most all the bugs out of our software before we ship it.Goodnight: No, that's a shortcut to disaster. We shipped a bad release back in 1981. We hadn't tested it sufficiently. We just finished coding it, did a few tests, packed it up and shipped it out. We had a lot of users who were quite upset with us because they found hundreds of bugs. It took us a couple months to fix [them]. We realized that we had wasted so much time trying to fix the bugs in the field that we made an absolute rule that this would never happen again.We don't announce ship dates of any of our products until we know when that date will be. We keep track of the bugs being found each week by our testing department. Only when the number of bugs drops back down to almost to zero will we ship a product.These things are funny. We have graphs maintained back to the mid '80s for every release. I can show the history of how many bugs were found with every release. First, you find more and more bugs each week. Suddenly you reach a peak, and then you start coming back down. In a major release of a system, we will find about 1,000 bugs a week at the peak. Every release is the same way.So until you are getting where you find fewer and fewer bugs each week, you're not ready to ship. A lot of companies ship when they are near the peak, when only about half the bugs have been found.