Working with Environment Manager ETL Logfiles and EmMon Log Viewer Filters

*This post originally appeared on the AppSense blog prior to the rebrand in January 2017, when AppSense, LANDESK, Shavlik, Wavelink, and HEAT Software merged under the new name Ivanti.

When Environment Manager 8.6 was released and the preferred method of logging changed to ETW (Event Tracing for Windows), like many others I initially used the older logging tools so that I could still create the text based logs I’d been used to working with since Environment Manager v7 (old habits certainly do die hard!).

Recently, however, I’ve been making a conscious effort to get a little more up to date, and where possible use the latest EmMon log viewer. This is part of the “EnvironmentManagerTools” installer, and includes the following main executables:

• ClientLoggingSetup.exe
• EmMon.exe
• EMPFileUtil.exe
• EMPMigrate.exe
• EMPRegUtil.exe
• Etl2Txt.exe
• FBRExplorer.exe

EMPFileUtil.exe, EMPRegUtil.exe, EMPMigrate.exe and FBRExplorer.exe and specific to User Personalization which I may cover in future blog if there is enough interest (I’ve written some scripts that support use with this to accomplish bulk export of data, and bulk deletion of registry data).

For this blog, I’ll concentrate on EmMon.exe (the log viewer), ClientLoggingSetup.exe (used to configure logging), and briefly touch on Etl2Txt.exe – a handy tool for those who want the performance improvements of ETW logging (Event Tracing for Windows), but still want to use their favourite text editor to view the logs.

Why ETW?

The move to ETW came about for a number of reasons. We often find that, when troubleshooting intermittent issues, a restart of the Environment Manager Agent (the “AppSense User Virtualization Service”) can resolve an issue a user is currently seeing – the problem here is Support need logs to investigate, but the process of getting those logs fixes the issue. You could leave logging enabled permanently, but herein lies other issues – disk space and performance. Whilst having logging running continually on a desktop OS may be manageable, on a busy TS environment this can be a challenge.

With EM 8.6, the ClientLoggingSetup.exe can be used to enable ETW logging without the need for an agent restart, plus there are options to enable circular logging and configure maximum a maximum logfile size. Additionally, logging can be enabled only for a specific component, such as EmUser.exe.

What you need to know

At first glance, the ClientLoggingSetup tool may look a little daunting, but once you know the main components of Environment Manager, you can (in most cases) ignore a lot of the options. The main components are:

EmCoreService.exe – This handles messages between the various components, and is ultimately responsible for launching EmUser, EmSystem and EmUserLogoff, and telling these components when to run various triggers (Logon, Logoff, Process Start/Stop etc.) You’ll often read or hear reference to “triggers” – these are events that EmCoreService gets from various sources and, where applicable, sends to the relevant process, which in turn applies the actions / conditions for the nodes within that trigger i.e. User > Logon

EmSytem.exe – This runs everything associated with the “Computer” triggers (startup / shutdown actions)

EmUser.exe – This runs all user logon and mid-session triggers (i.e. process start / stop, session lock)

EmUserLogoff.exe – This runs all user logoff actions, including process start/stop actions that run during logoff, and sync-up of Windows Settings Groups and any managed applications left open at logoff. When a user logs off, EmUser.exe exits, and operation is passed to EmUserLogoff (unless an open application allows the user to cancel the logoff, in which case a new EmUser.exe process is launched).

Image 1 – Client Logging Setup

I won’t go into all the options in the ClientLoggingSetup as these are covered in the documentation in “AppSense Environment Manager Administrative Tools Guide.pdf”. However, the following options you should be aware of:

• File size limit - The maximum size of the log file in Mb if Circular Logging or Live Logging is enabled.
• Rollover log - The log file grows to the specified size limit. Once it has reached the limit, a new log file is created with a version number appended to the name.
• Live log -    Logging can be viewed in real time using the Environment Manager Monitor. This is the equivalent of Real Time logging in Windows Event Tracing.
• Circular log - The log file grows to the specified size limit. Once it has reached the limit, the log file automatically overwrites the oldest entries.
• Unlimited log - The log files grows indefinitely regardless of the size limit.

The LogViewer (EmMon.exe)

So this is the main point of this post, and although this does not cover everything, I do aim to point out some handy little tricks and useful pointers – if you would like me to go into more detail just leave a comment and I may do a more in depth post on this alone.

The screenshot below shows the opening view of EmMon, and also my first tip – many of the log entries I look for in my filters use loglines that are set at the “Trace” loglevel – if you leave the default detail level, EmMon will not load these log lines from the ETL file. You can enable them after a log file is loaded, but you will need to click the folder icon next to the “Recent log files” drop down to reload the file and parse the missing trace entries. To load the trace entries, click the “Include detail levels” drop-down and check “Trace”. Additionally, troubleshooting logon, for example, may also need you to change the “Include event types” to “All” so that EmCoreService entries are loaded.

Image 2 – EmMon initial view

Again, the “AppSense Environment Manager Administrative Tools Guide.pdf” file covers usage of most of the basic features, and so I’ll be expanding on the “View/Search all detailed logging” option, rather than the quick actions (Analyze user activity, Find trigger bottlenecks etc.)

Logon is more than just Logon

One of the most common titles for support cases has to be related to “slow logon” – but before diving in to the logs, what is logon (in terms of AppSense).

Logon actually covers a LOT of activity, even just in terms of Environment Manager. Obviously there are other things happening (Group Policy, Logon Scripts, Startup programs etc.), but from an Environment Manager perspective, all the following (where applicable) take place:

Personalization:

• Contact Personalization Server to download site list (initial poll to send client details to be checked again membership rules etc., and return a list of servers for the relevant site)
• Contact Personalization Server to download configuration (list of managed applications, includes, excludes, and other settings)
• Download Windows Settings Group data, evaluate Windows Settings group rules, apply settings and perform any required desktop refreshes.

Policy:

• Populate session information
• Launch EmUser.exe into the session
• Run Pre-Session trigger then Pre-Desktop Trigger
• Once the shell program is operational, run the Desktop Created trigger

Image 3 – Environment Manager Trigger Nodes

Also, it is worth bearing in mind that while Process, Network and Session triggers are not listed under “Logon”, they are trigger based, and as such will run during logon if required – so every process start/stop that occurs during logon will be checked against all process start/stop nodes & conditions.

The first part of troubleshooting logon / logoff, is to make sure EmCoreService handles an event that the user is indeed logging on or off, it launches the appropriate (EmUser / EmUserLogoff) process and that process gets the event from EmCoreService.

The following filter gives a very simple view of this whole process:

Image 4 – Session Detection

Here I’ve changed the default root level “operator” in the filter to “or”, then clicked on this “or” item to add several “Groups”. Following this, I also used “View” > “Columns” > “Column Chooser” to remove some of the noise – leaving me just the “Time Stamp”, “Component”, “Session Id” and “Raw Log Line” columns – this is then much easier to copy/paste (Edit > Copy as Text) or, if you want to refer back to this list but create another filter, use “Edit” > “Copy to new tab” – this is a very useful feature as you can flick between different views of log lines you’re interested in without losing the results of a given filter (Note: the new tab is just the loglines you selected to copy, it’s not a filtered view of the entire file so isn’t affected by clearing / editing the filter).

For very little log entry, this actually shows a lot of information. The first 4 lines show EmCoreService getting it’s notification that a user is logging on, launching EmUser.exe and telling EmUser to start running the “Pre-Session” trigger, internally referred to as “EarlyLogon”.

The remaining log lines show a lot of entries that start with [EmTask … and end with either ENTER or EXIT. From these we can see how long each main “trigger” took to run.

All the actions for an entire trigger are contained (chronologically) within an “EmTaskRun … ActionsEE” block, however, the internal log lines differ slightly in naming convention from the more “friendly” names used in the Environment Manager console – below are some of the more common ones:

Something that will quickly become apparent if you’re reading this, is that there is currently no mechanism to load / save Filter settings – during writing this article I raised this as a feature request with Product Management. If you also believe this would be particularly beneficial please raise a feature request.

Okay, so that’s the policy aspect of logon / logoff, but we may also have Personalization to consider.

The following filter example shows log entries for EmUser.exe related to a number of key points:

• Download Personalization Server site and config details
• Run the 3 logon related triggers
• Evaluate Windows Settings Group Conditions and restore the data
• Set Icon Positions

In terms of logs, you’ll notice another difference between naming conventions in logs and the console – the relevant log entries are “EvaluateDSGConditions”, “CDesktopSettings” and (after the shell program starts) “IconPositionManager” / “RestoreIcons”. In this example, I’m only interested in the Enter / Exit (Start / Stop) times for the main functions, so I have two filter “Groups” and each log line must contain one entry from each group – i.e. “Show me lines containing (enter or exit) AND (any of these functions)”

If you really, really need text logs!

For those that really, really want to stick to good old text editors and scripts, another utility with the Environment Manager (admin) Tools it Etl2Txt.exe – as it names suggests, this will convert an ETL file to its text based equivalent files, so you can collect the logs in ETL format and have all of the wonderful goodness above, but also use your old favourites when needed.

It’s a command line tool, so I find adding a shell command for the explorer context menu makes life a whole lot easier – in my case I already had a handler for ETL files to use Windows Performance Analyzer, so I simply added an extra option within here, if you do not have this already, refer to the following Microsoft article regarding the addition of custom file associations, using the “command” entry detailed in my steps:

In my case, I added two new registry keys:

HKEY_CLASSES_ROOT\wpa.etl_file\shell\convert

HKEY_CLASSES_ROOT\wpa.etl_file\shell\convert\command

In the first one, set the “default” value to, for example, “Convert with ETL2Txt”

In the second one, set the default value to the path of Etl2Txt, passing the –f and –o parameters (source file / output folder):

C:\Program Files\appsense\environment manager\tools\etl2txt.exe -f "%1" -o "%w"

In terms of .reg file, this would look like the below:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\wpa.etl_file\shell\convert]

@="Convert with ETL2Txt"

[HKEY_CLASSES_ROOT\wpa.etl_file\shell\convert\command]

@="C:\\Program Files\\AppSense\\Environment Manager\\Tools\\etl2txt.exe -f \"%1\" -o \"%w\""

If there is a particular area of the Environment Manager Tools, or the log files themselves, that you’d like more details on, leave a comment and I’ll happily write a further article as required.