Thank you to Hyperion Unused Member Analyzer testers, and thoughts on future tools

Thank you all so much for helping out. I am absolutely blown away at the response that this utility has generated from all of you. Please let me know if you run into any issues.

Changing subjects (and zooming out) a bit, back to my efforts to understand what you (as consultants and Hyperion professionals) check during your “health check hit list“, it’s my goal over the next year to put together a suite of power tools that enable all of us to create, analyze, and maintain more robust solutions. HUMA is one such tool in the toolbox.

I have a few other ideas up my sleeve, but if you ever find yourself saying, “Self, I wish I had a tool for [fill in the blank]” or “I wish there were an easy way to…” then I would love to know about it. Even if it’s something you do already that’s perhaps manual and laborious, perhaps it can be automated, sped up, improved, and made useable by the community at large.

Beta testers wanted for Hyperion Unused Member Analyzer tool

I have been working on a tool called HUMA – Hyperion Unused Member Analyzer. The idea for it came out of some side discussions at Kscope a couple of months ago. The idea is simple: Wouldn’t it be nice if there was an easy way to determine if any members are unused in a given cube?

Given a server, database, and credentials, HUMA will connect to a a cube, analyze its stored members, generate a list of all possible values, then iterate over it, analyzing the resulting data grids for the presence of data. If there are members with no data in them, they are shown to the user running the program. To increase performance, HUMA orders the grids and sequences of members within the sub grids so that they are aligned to the dense/sparse structure of the cube, so that it can pound on the same hot blocks before moving on to grids with different sparse permutations.

On a pretty gutless VM of mine with Essbase running in 1GB of RAM, a standard Sample/Basic cube can be ransacked for data in about three seconds. Also, given the way the tool works, it’s not necessary to do a full export of a cube or anything since the analysis is based on the data that is queried and immediately discarded. So far it seems to work pretty well.

The goal of the tool is to be a tool in the toolbox for Hyperion/Essbase admins that want to analyze their environment and act on possible improvements. This goes hand in hand with my research and efforts to find out what we all do when we dive into a new system as part of a health check hit list.  Doing so on a BSO database can yield improvements (particularly on dense members).

In any case, version 1.0 of the tool is basically ready to go and I’d love to have a few people test it out and let me know of any issues!

Eclipse tips for you Java folks out there

I know there’s lot of you Java and Essbase folks out there in addition to myself. I’m always interested in ways to use an IDE more effectively. I came across this list of Eclipse annoyances and easy solutions for them earlier today. I am already using some of these but some of them are new to me. So if you want to get your environment dialed in a little bit, check these out. And for you IntelliJ folks… just keep being your normal smug selves. ;-)

Oh, and I’m thinking about doing a small series on “Essential Java for Essbase Developers” or something along those lines. It’d be something like a tutorial series on how to navigate around all of the Java terminology, features, and ecosystem in order to get straight to developing the solutions you want to develop, just a little bit quicker. Let me know if you have any thoughts or suggestions!

 

 

Processing a file line by line in Java

I seem to use this pattern a lot, particularly in projects without dependencies (i.e., no Guava). What’s the cleanest way to iterate through a file line by line in Java? There are a lot of examples that are pretty messy. Here’s one I keep coming back to that is nice because it uses a traditional for loop to manage things. In this case it’s not just a counter variable but instead, a more general version of the for loop where we have an initializer, while condition, and iteration statement.

public void process(Reader reader) throws IOException {
  BufferedReader bufferedReader = new BufferedReader(reader);
  for (String line = bufferedReader.readLine(); line != null; line = bufferedReader.readLine()) {
    System.out.println("Line: " + line);
  }
  bufferedReader.close();
}
That’s it. That’s about as clean as I think I can get it. Note that the argument to the method isn’t a String filename but rather a Reader object, which alleviates the need for this method to have to catch FileNotFoundException as the caller would presumably supply a FileReader or some other type of Reader.

Hyperion code drop: Parent Inferrer tool

I am pleased to release another small one-off Hyperion-related open-source tool into the wild: Hyperion Parent Inferrer. This Java program/library can be used to translate a space-delimited hierarchy file into one with explicit parent-child mappings. For example, consider the following input file:

Time
 Q1
  January
  February
  March
 Q2
  April
  May
  June
 Q3
  July
  August
  September
 Q4
  October
  November
  December

Hyperion Parent Inferrer will generate the following output:

null,Time
Time,Q1
Q1,January
Q1,February
Q1,March
Time,Q2
Q2,April
Q2,May
Q2,June
Time,Q3
Q3,July
Q3,August
Q3,September
Time,Q4
Q4,October
Q4,November
Q4,December

As with most of the things I release these days, this tool is just a cleaned up version of something that I needed or used once or twice. In this case I wrote this up to help with a conversion process at some time in the past.

Now, I’m not saying that the approach I had to take was ideal or there weren’t better solutions. But oftentimes the environment that we operate in is less than ideal and you just gotta find a way to duct tape everything together. Hence this tool. The code as it sits today works perfectly fine but does not provide much in the way of configurability or options that might make it useful in more contexts. But as with some of the other projects I have released, I thought it would be nice to toss this over the fence in case anyone can benefit from it.

If you find a use for this or want to show me your clever Python/Perl one-liner that does the same thing, let me know!

ADS File Reader 1.0.0 released

As promised, I cleaned up the Java code for very easily parsing through files in the ADS format. ADS File Reader has an incredibly intuitive, clean, and functional API that makes parsing through ADS files a breeze. You won’t believe how easy it is to burn through an ADS file (if you have to). I can’t imagine that there are a ton of people out there that need to do this, but if you wind up having to (and especially if you need it in a larger Java program) then you should be all set. As with most of my other projects, ADS File Reader is licensed under the Apache Software License version 2.0 and is available on GitHub. For more information and a link to GitHub, check out the ADS File Reader project page.

Thanks to all of you that sent in some ADS files that I could play with. It definitely helped flush out a couple of kinks. If you end up using this library please let me know, and definitely let me know if you run into any issues.

Do you have any ADS files you can share with me?

As you may have noticed I have a penchant for throwing my various Java and other language endeavors over the wall and seeing what happens. Most of these are open source programs ostensibly in the Hyperion Essbase space, but every day my focus seems to broaden a bit. I’ve open sourced data generators, validation tools, substitution variable updaters, and more.

The particular code that I am cleaning up now and getting ready to kick out as a small Java library for reading the ADS file format. Awhile back it was necessary for me to work with raw ADS files and use them as the input to another system. For those of you that aren’t familiar, the ADS file format is used by LCM when you kick out an EPMA dimension. I’m sure it’s used in other contexts too but this is just what I know it from. It’s not incredibly complex but it does have its nuances.

I’d like to be able to test it against some files other than some of my own, so if you have an ADS file laying around or can generate one that you can share with me so I can test with that too, I’d appreciate it. I will absolutely NOT distribute it with the rest of the program or otherwise share it with anyone else unless you explicitly say I may do so.

Thanks!

Essbase Java API jars in Maven

This is a bit of a repost with respect to content as I’ve already talked about installing Essbase Java API JAR files into a local Maven repo. The other day I installed the newest JAR files for 11.1.2.3 and thought I’d do a quick post. There doesn’t seem to be a “CPLD” JAR anymore – I’m not totally sure what it did but some googling awhile back suggested we don’t need it anymore. So that leaves the main JAPI JAR and the server JAR:

ess_es_server.jar
ess_japi.jar

I’ve renamed them as thus:

essbase-es-server-11.1.2.3.jar
essbase-japi-11.1.2.3.jar

Then the following two Maven commands:

mvn install:install-file -Dfile=essbase-japi-11.1.2.3.jar -DgroupId=com.essbase -DartifactId=essbase-japi -Dversion=11.1.2.3 -Dpackaging=jar

A whole ton of output will show – hopefully some version of a sucesss message. And then this:

mvn install:install-file -Dfile=essbase-es-server-11.1.2.3.jar -DgroupId=com.essbase -DartifactId=essbase-es-server -Dversion=11.1.2.3 -Dpackaging=jar

And again a bunch of output. If you use Eclipse like I do then you might want/need to rebuild your local Maven repository index so they are picked up there, then you can add them via the GUI if you’re so inclined by just filtering on “essbase”. Add them both to your project and then go from there. You might need some other dependencies for the new Oracle Diagnostic Logging (ODL) which seems to have been implemented. I haven’t played with this much but it seems to depend on the version of the server you use. I typically use SLF4J and bridge the Essbase JAPI logger, but with ODL things might have changed a bit. I’ll post an article at some point hopefully discussing a resolution. If you bump into any issues or solutions please let me know.

Evolving the Essbase Java API with JALE

I work with the Essbase Java API quite a bit. In fact, it’s more or less the reason that I picked up Java oh so many years ago. If you’re writing an app to do custom queries against a cube, need to develop a middle tier for your mobile app, or want to pull some cube stats, it’s the place to be. For me it’s a lot more useable than the C or VB APIs.

That being said, working with it can be a little challenging sometimes. Not because of inherent complexity, but because it was designed quite some time ago (especially in technology years) and before a lot of Java niceties came to be. In particular, there are iterators and other constructs that require a fair bit of boilerplate code to deal with. Here’s an example of getting a count of connections from the server:

public int getConnectionCount2(String username, String password, String server) {
	IEssbase essbase = null;
	IEssOlapServer olapServer = null;
	try {
		essbase = IEssbase.Home.create(IEssbase.JAPI_VERSION);
		olapServer = essbase.signOn(username, password, false, null, "embedded", server);
		IEssIterator connections = olapServer.getConnections();
		IEssBaseObject[] connectionInfoObjects = connections.getAll();
		logger.info("Connection count: {}", connectionInfoObjects.length);
		return connectionInfoObjects.length;
	} catch (EssException e) {
		logger.error("Essbase error: {}", e.getMessage());
		return -1;
	} finally {
		try {
			olapServer.disconnect();
			essbase.signOff();
		} catch (EssException e) {
		}
	}
}

Wow, that’s a lot of code – just to do something that’s ostensibly simple. We can’t realistically expect Oracle to overhaul their API. While it’s technically possible to create a new API (from Oracle’s standpoint) that’s also unlikely to happen. Unwieldy though it may be, it’s battle-tested and thorough (although a few of you out there are miffed that it doesn’t have support for varying attributes).

So if we can’t expect the API to be refactored and offer us some syntactic sugar, and can’t write a new API, what can we do? There’s another approach: wrap the API with another library with the express intent of making the Essbase Java API easier to use, less verbose, and leverage modern language features where it makes sense.

To that end, a side project of mine has been to create such a library, called the Java Antikythera Layer for Essbase (JALE). Here’s an example of some new code that can iterate the current Essbase connections. Note that this isn’t a direct replacement for the above since they do slightly different things:

@Test
public void testGetConnections() throws Exception {
	try (OlapServer server = Ess.create().signOnEmbedded(SERVER, USER, PASS)) {
		for (OlapConnectionInfo connInfo : server.getConnections()) {
			System.out.println("Conn info: " + connInfo);
		}
	}
}

Note that this is taken from a unit test in one of the projects I have for this little experiment of a library. As the library evolves more or as more people are interested I’ll post more samples and open up a GitHub repository. Some things to note about the above code-snippet and the design of the wrapper layer in general:

  1. Adds a convenience signOn method (signOnEmbedded) to make the intent of the method instantly understandable versus passing in a parameter called “embedded” as the current API requires.
  2. Drops the IEss prefix on classes/interfaces. Modern IDEs have reduced the need for this quasi-Hungarian notation.
  3. Instead of returning an an array of objects that need to be iterated and cast to their real types, a List of the expected object is returned (server.getConnections() returns the list).
  4. Java 7’s new “try-with-resources” construct is used and the JALE server object implements AutoClosable, obviating the need for a finally block here.
  5. Also note that JALE wraps the EssException in disconnect/signoff methods so we don’t need a dummy try block inside our finally block.
  6. Ess.create() is a convenience method that auto passed in the IEssBase.JAPI_VERSION static field. We can use an overloaded method if we need to call it with that parameter.
  7. The more astute of you out there may notice a couple of other things. :)

Internally, JALE has a utility library designed to help ease the implementation of replacement classes and call the right methods. Of particular note is an “IEssIterator” unroll method that is a veritable Swiss army knife for easily replacing the boilerplate code around the Essbase iterator. For example, check this bad boy out:

public class IteratorUtil {
	@SuppressWarnings("unchecked")
	public static <E, F> List<F> iteratorToList(IEssIterator iterator, ConversionDelegate<E, F> delegate) throws EssException {
		List<F> objects = new ArrayList<F>();

		for (int index = 0; index < iterator.getCount(); index++) {
			E object = (E) iterator.getAt(index);
			F converted = delegate.convert(object);
			objects.add(converted);
		}

		return objects;
	}
}

&#91;/java&#93;

Then in our wrapper for the traditional Essbase server object, we can do this:

&#91;java&#93;
public List<OlapApplication> getApps() throws EssException {
	return IteratorUtil.iteratorToList(olapServer.getApplications(), new ConversionDelegate<IEssBaseObject, OlapApplication>() {
		public OlapApplication convert(IEssBaseObject from) {
			return new IEssOlapApplicationWrapper((IEssOlapApplication) from);
		}
	});
}

Note that conversion delegate is a one method interface that just supports the generified convert method. It’s designed so we can just supply a lambda-esque convertion method block in the new method (in this case, getApps()) which in this case contains our logic for going from an IEssbaseObject to the designed type of OlapApplication (a new object in JALE).

Future Directions on JALE/a Hyperion Essbase API wrapper

There’s no timeline on a public release of JALE. At this point it’s highly experimental as I evaluate its feasibility. I am just implementing method as I need them, so don’t expect to see me wrapping the MDX routines any time soon (not that I don’t use them but it’s not a section of the API that “needs” much fixing). I’m not worried about a performance hit from wrapping the API but I do have some concerns about performance if there’s something in the native API that loads lazily or otherwise needs an active connection to the server that would cause things to act up when I wrap them an enumerate on them, for example.

As with all of my little side projects (particularly the open source variety), please let me know if you have any feedback or want to contribute. Unlike the others at this point, this one is very pre-alpha quality so I don’t have immediate plans to make it available, but jut wanted to discuss some of the things I am thinking of an working on.

Announcing Jolo, a Java library for printing text-based tables

Well, I’m at it again releasing another open-source project. I work with Hyperion and the Essbase Java API extensively, so it’s not uncommon for me to be doing things with grids and working on a command-line. I frequently write lightweight command-line clients to test things out. I didn’t love the options I was able to find in Java that would help print/format a text-based table. The ones I did find were clunky and hard to use.

So, you guessed it, I rolled my own. Jolo is a small and lightweight Java library with two goals: print out text-based tables, and have a tight and clean, yet flexible API. So let’s say we want to print out names, cities, and states in a three column table. The code looks like this:

public void testPrintTableWithColumns() {

    // setup table definition with column names/widths
    TableColumnList tcl = new TableColumnList.Builder()
        .add("Name", 40)
        .add("City", 15)
        .add("State", 2)
        .build();

    // createRandomRows is a helper function in this case but would otherwise
    // be your data that is an Iterable<List<Something>>.
    Iterable<List<? extends Object>> data = createRandomRows(10, 3);

    // create the printer and print the data
    TablePrinter tp = new TablePrinter();
    tp.outputTable(tcl, data);
}

Having the TablePrinter separate from the concept of a TableColumnList is nice because we can plug-in different TablePrinter implementations if we want to. In the above example I have a helper method creating the data for me (createRandomRows()) but in your program this would be something that implements the Iterable interface and contains a List of something that extends Object. In Java parlance that means you can print anything that’s Iterable<List<? extends Object>> – note that the toString() method will be called, so you can pass anything in. If I get time I’ll enhance the API a bit to provide some convenience functions and syntactic sugar. Given some data, the above code generates this table:

+----------------------------------------+---------------+--+
|Name                                    |City           |St|
+----------------------------------------+---------------+--+
|Jason Jones                             |Seattle        |WA|
|Cameron Lackpour                        |Philadelphia   |PA|
|Tim Tow                                 |Huntsville     |AL|
+----------------------------------------+---------------+--+

Note that with a simple parameter in the builder we were able to constrain the width of a given column.

The Hyperion Connection

This isn’t ostensibly Hyperion- or Essbase-related (save for the names in my table, hehe), but if you work on the things that I work on then this might be up your alley. The Jolo page has more information including a link to the Github repository. I will likely push this to Maven Central as time permits to make its inclusion in anyone’s projects all the easier. Unlike many of my other projects, this one is not incredibly commented [yet], so that will be coming in the future weeks as time permits. The API is pretty clean though so you shouldn’t have a hard time using it. Licensed under the very business-friendly Apache Software License.