Dynamic Cubes was the word of the day for me. It’s an exciting new feature of Cognos that lets you build cubes on top of relational systems. By using these cubes, you can get tremendous improvements in your report runtimes. The method of building them seems fast and easy, and while I have a few minor misgivings about some of the design decisions, I am really looking forward to getting an opportunity to play with them.

At the most basic level, Dynamic Cubes are an extension of the DQM engine. As I’m sure everyone in the world is aware, DQM is the 64bit querying engine released with Cognos 10.1. They are based on relational data sources, and work by caching the contents of your data warehouse.

First the cube needs to be modeled. I didn’t get a chance to see the cube modeler, but the developers are saying it looks and feels very similar to a cleaned up Transformer. Attributes can be defined (sadly lacking in Transformer), and dynamic dates are easy to build. It is important to note that the new modeling tool is far less forgiving for nonsense than Transformer. MUNs must be unique, or it will throw an error. The data warehouse must be set up in a star schema or a snowflake for the cube to work.

Once the cube has been built and published, it needs to be started. After it starts up it will start building the various caches. The member cache consists of all of the members in the dimension tables. Aggregation caches are populated as the cube runs. These can be contextually aware aggregates, so the cube will benefit if you have aggregated tables. The data cache is populated as reports are run. If a report has been run before, it will be in the cache and will be rendered instantly. An expression cache exists – any matching expression will be populated instantly.

As reports are run, it will be possible to determine if the aggregations used are optimal. The dynamic query analyzer has a new option for the cubes. You can have it check the run history for specific reports, or users, and it will optimize the caches aggregations to ensure the best performance.

The developers responsible for this innovation referenced a case study in which a report that took roughly a day to run went down to 3 seconds. From what I understand, the data warehouse was the same, but they needed to add some more ram.

The dynamic cubes take a fair amount of ram. IBM will release a white paper discussing the various sizing and ram requirements.

It seems that real life is creeping it’s way back in. Today was the final day of the Expo, and unfortunately I didn’t have a chance to meet with all of the people manning the booths. Over the next week I’ll try to go over the remaining documentation that I pulled from various booths.

Lots of interesting sessions today. I learned quite a bit about the Dynamic Cubes, which sounds phenomenal. And, as usual, got lots more swag and books.

The biggest treat for me today was a chance to meet and chat with some of the Motio people. A big shout out to Lisa, of LisaLovesMotio fame who is absolutely wonderful.

I had the chance to sit with Motio’s Adam Bonneau, who is a Senior Software Engineer, and Holly Rice, whom I’m sure everyone who is reading this is familiar with. Adam was gracious enough to show me an amazing piece of software he was working on that will revolutionize the way we test our reports in Cognos. Sadly, we only had about an hour to go through the software, so some of my information may be slightly off, especially the terminology.

Sadly, this article will be lacking in screenshots as I haven’t yet been able to beta-test the tool. The request is in, and if (when, right guys?) the tool does go into beta, you can be sure there will be screenshots left and right.

When he first loaded the page, I thought at first it was a local client tool. It is entirely webbased, but it acts and behaves like a thin client. The testing tools can be accessed in two ways, via the main application or through report studio. But before I can go into the tools themselves, a few words about the theory behind the tool.

Motio is building MotioCI with the concepts of assertions. The assertions essentially indicate what you are expecting from the report. This can be expectations before, or after the report has been executed.

An example of non-executed assertion would be checking naming conventions, or if reports are missing specific logos, or if you have queries that aren’t being referenced by any data containers. All of these can be determined by looking at the report XML.

Execution based assertions would be ones that look for specific elements in a report. When you run the report, you’re expecting to see the same values in a list as a static SQL statement. Or you expect two separate lists in two separate reports to have the same order of elements. You might also be looking for average run time, or run time variance.

When the user runs the testing tool, either from Report Studio or directly from MotioCI, you can select a list of predefined assertions to test. The predefined assertions are based on best practices, common corporate policies, and simple common sense. As each report, or dispatcher, or workspace, or package, or any other object in Cognos is tested, it will create a list showing the results of the status of each assertion. Failed, Succeeded, or Stale.

Clicking on Failed will show you exactly why it failed. If a specific query is causing local processing, it will tell you the name of that query. If there is a malformed list or crosstab, it will give you the name of that object.

Stale simply means that it has previously tested the report, but the report has changed since.

All of the data from the testing is saved in a table, which makes it extremely easy to burst a report to the authors and owners of the objects informing them of any problems that have been encountered. This will very quickly allow enterprises to ensure that every single object meets corporate guidelines.

By that point, I was already salivating at the idea of trying it on my own, but then he showed me Assertion Studio.

Assertion Studio, as the name suggests, allows admins to build their own rules. The complexity is astounding, and I strongly suspect that Motio could easily recoup their development costs on training in this studio alone.

The best way to describe Assertion Studio would be to picture a work flow. The report runs, you locate a specific list named VARIABLE, you tell it to find the first n rows, you run an SQL statement that returns the same number of rows, you compare the two result sets.

Or you open a specific report, and do an xpath search for a specific element, and a regex search for a specific pattern. All drill through definitions (element) pointing to \My Folders (pattern), for example. You can say if it finds the element, but doesn’t find the regex pattern then fail it. Or you can add another set saying it finds the element and it finds the regex then do something else.

Additionally, you can also do impact testing from inside MotioCI. From MotioCI you can say that you want to promote your package and a few reports. The tool will automatically determine which reports will be effected by the promotion, and it will allow you to specify which assertions you want to test. Similar to life cycle manager, it will test both versions with the assertions, on the target machine with the same credentials and data source, and allow you to see the results side by side. The difference is that you can very easily see the results of the assertions on the original and new reports. So you can see that the average run time on the original report was 5 seconds, while on the updated it’s only 3 seconds.

There was so much more to this tool, I feel like I’ve barely scratched the surface.

Sadly, I’m going to have to end this post before I detail all of the other wonderful people I’ve met in the Expo or sessions. But I will say that my pen count has increased to about 18. One of them is shaped like a rocket!

Good news! I am now an officially certified IBM Cognos 10 BI Author and IBM Cognos 10 BI Multidimensional Author.

There are some very interesting new technologies coming out of the IOD this year. The most impressive to me are the PureSystems and Dynamic Cube. Also the support for Bi-Directional in Cognos. Did I mention the bi directional support? Cause that’s awesome!

The PureSystem and PureData are preconfigured boxes that come with everything you need for a BI platform. OS, Database and a central management system that covers it all. Cognos and SPSS needs to be installed on another server, but the performance is supposed to be phenomenal. The scenario I gave them is based on the needs of one of my clients. With 200k rows inserted into tables per second, the real time reports their customers need are unacceptably slow. The IBM guys assured me that not only can Netezza handle such drastic bulk loads, the system will return the results of any query quickly and efficiently. I’m trying to set up a demo to see if it’s something my client will want to work with.

Dynamic Cubes are a new thing in Cognos 10.2. It has it’s own development studio to build them, which is supposed to be similar, but simpler, to Transformer. It can support multiple datasources by creating multiple cubes, then a virtual cube to combine them. Apparently the way it works is that you first design the cube based on your data source, then as users run the reports it will cache the results to make future queries faster. Since it’s a honest to god cube, and not DMR, it fully supports all the dimensional functions that you would expect from an OLAP source.

I got my own signed copy of 5 Keys to Business Analytics Program Success. Thanks to all the authors who were there to sign it!

The various people manning the stalls at the Expo were all wonderful and very willing to talk about what they do. There were lots of very interesting technologies and solutions on display. In no particular order (pulling them out of the bag as I do this):

1. Vormetric provides data security by transparently encrypting databases. They gave me a pen and a nice business card case!
2. Vega has The Vega Unity Suite which allows you to integrate multiple repositories and search through them on multiple browsers or devices.
3. Viewpointe does Information Governance, and was giving out awesome footballs.
4. Microsoft had a stand there that was talking about the Open Data Protocol. It seems like a very interesting way of retrieving data from a database.
5. Datacert is a solution provider with some very nice demos, plus they gave me a Rubix cube and a nice pen.
6. eCapital was giving out funky digital clock/card holder/little bendy toy man thingies. It’s fun. They do Business Analytics solutions.
7. Kinetek Consulting does consulting and solutions. Very nice demo. They were giving out nice paper pads.
8. ImageSource does ECM services and consulting. Unfortunately that’s not really my area but they had an impressive stand. They were also giving out nifty rave style blue led light stick thingies.
9. Nice had a really awesome demo set up. The demo was showing the progression of a frustrated customer calling a call center. They were able to match the activity of the customer from clicking on the ad (of which they could call up), to him calling to complain about something. The demo showed how managers and supervisors can use their system to track the agents behaviour, and the customers sentiment. From this system you were able to hear the conversation, send emails to the supervisors or agents, and view the entire progression of the contact. Of course it also showed you the standard call center metrics, but everything can do that. Scored a pen off them.
10. ESRI. Esri is a lot of fun. I’ve always had a soft spot for geoanalytics. Ever since I heard of the first use involving tracking cholera in London. Their maps are awesome and wonderful. They were giving out the Esri Map Book showcasing the most impressive use of their maps. Looking at it now I continue to be blown away by what they can do. They also gave me a pen! Now go here and enjoy their maps. Then come back, there are more pens to talk about.
11. VirtUSA does consultations and solutions. They had a decent demo there. They were giving out nice cloths to clean screens, useful on glasses too.
12. ADLIB will convert anything into PDFs. The system will allow you to create chapters and all sorts of nifty things. They were giving out pins.
13. Aviana Global had a very very nice stand. And a motorcycle that they’re raffling off. And motorcycle video games. I’m just wondering how I’m going to convince AA to allow me to check the motorcycle as luggage when I win it. They gave me a pen, and a foam bottle holder that entitled me to a free beer from the complimentary bars. Double free beer!
14. Viewpointe were showcasing their information governance platform, OnPointe. It looked impressive but unfortunately it’s not really my area. They were giving out some nice pens.
15. BSP are wonderful people and I’m not only saying that in hopes of getting a shirt. Their stand was right next to a bar the first night, and next to a lunch stand today. They were showing off their always impressive meta manager and version control suite. The big plus for me was that I finally got to meet the wonderful people there (really, I’m not saying this just to score a shirt). Mark, also known as MFGF on the Cognoise forum was there as the BSP Cognos Champion. So kudus for him! Note that I’m not at all jealous of him. Not in the least. They were giving out what I first thought was a cheese grater for the iPhone, but turned out to be a nifty pen holder.
16. PerformanceG2 was a fun place to visit. Really nice people, plus I got to meet the very lovely Candace who is absolutely prolific in the LinkedIn and Twitter world. They also gave me a very nice pen.
17. Peritas Solutions is a Calgary based consultancy/solutions company. I chatted with them for a bit, they’ve been working with Cognos a long time, back to when it was owned by a company called Cognos, and the product called PowerHouse.
18. DATA41 also does consulting and solutions. Very nice people, they were almost throwing candy at me at one point. They also had some really nice flasks to give out that fit perfectly into the IOD badge case. Take a drink into the sessions and noone will be the wiser!
19. Integritie was the company whose name I forgot last night. I may have been delirious from sleep deprivation last night, but I do remember that they are wonderful people. They gave me a 250 meg flash drive that contains datasheets for their software. I’ll take a look at those later tonight.
20. Ernst and Young were there. They are also offering consultations and solutions. Apparently they have quite a few experts on staff who can do amazing things. The pen they gave me was the best of the night. Short but shiny and will act as a stylus – works very well on my Galaxy.
21. The Ironside Group was there. I didn’t talk to them much, besides telling them how much I loved their blog. Lots of tech tips there. They gave me a set of headphones, I think it has a microphone too. I’ll have to try that out.
22. Sky Solutions were demoing some very nice dashboards. Many of the things I do for my clients through heavy use of HTML items and JavaScript, they have prepared. Great people too, they had three representatives there, and we were all chatting for nice long time.
23. Datamatics also does consulting and solutions. Nice demos, plus they were giving out bags. Very very useful for carrying around all the stuff people kept on giving me.
24. IBM was giving out shirts at the infosphere area. I also managed to snag an IBM from somewhere, but I don’t remember where. They were talking about some very interesting things. I was later told that some of the people there were actually responsible for getting the bi-directional stuff in Cognos. I’m need to go back and shake each and every one of their hands until I find the one responsible and buy him a round of beer from the complimentary bars.

Total pen count as of today: 9

And that is all I can remember from today’s expo. Tomorrow I’m planning on attending the general sessions, learning more about Dynamic Cubes.