Set up Synthetic Monitoring with Internet Explorer in a "Headless" mode.
Synthetic Monitoring with IE has a lot of requirements that will need to be filled. Sometimes there are choices. Some easier than others.
Note: Although IE cannot run in a truly headless environment, it is possible to bind a desktop session so it remains open once you are disconnected.
Configure
Prerequisites
Available in Germain 8.6.13+
Tested with Internet Explorer 11
Engine will need to be run in a desktop session
User that runs Engine must be an Administrator
Host System should allow PowerShell scripts to run
Setup a Dedicated Engine Manager / Engine Instance for running the IE Monitoring
Same as setting up any engine (Engine Deployment), Only difference is it will only run the Dedicated IE monitoring on it.
If you have more than one Engine Manage on the System, you will need to specify a separate port in the Germain GUI (See section “Setting a different port for Engine Manager“)
Remember to set a unique name in germain-bootstrap.properties (germain.bootstrap.node="Node Name")
Internet Explorer cannot be run “headless”
So we will need to set up the engine to run in a desktop session.
Solution
Run the Germain in an active desktop session.
This is accomplished by use of the command tscon which we can use to hand off the RDP desktop session we use to connect to the Engine host.
Here is a PowerShell script that will clean up lingering processes from previous runs of the Engine, launch the engine, then hand off the the RDP desktop session (which will disconnect your RDP session)
You will need to edit this script to define the location of the dedicated IE Synthetic Monitoring Engine
start-active-engine.ps1 (WIP, need variables ser for Engine location etc..)
POWERSHELL
### Make sure we are administrator
#Requires -RunAsAdministrator
### Location of dedicated IE Synthetic Monitoring Engine
$EngineUser="germain_engine"
$EngineLocation="C:\Germain\GermainEngineIEMonitor\"
### Clean up related engine & manager
$nodes=@(Get-CimInstance Win32_Process -Filter "name = 'Java.exe'" | Where-Object {$_.CommandLine -like "*apmname=ie_syntheticnode*"})
$nodes | ForEach-Object { Stop-Process -ID $_.processID }
Start-Sleep -s 5
$nodes=@(Get-CimInstance Win32_Process -Filter "name = 'Java.exe'" | Where-Object {$_.CommandLine -like "*apmname=ie_syntheticnode*"})
$nodes | ForEach-Object { Stop-Process -Force -ID $_.processID }
$engns=@(Get-CimInstance Win32_Process -Filter "name = 'Java.exe'" | Where-Object {$_.CommandLine -like "*apmname=ie_syntheticengine*"})
$engns | ForEach-Object { Stop-Process -Force -ID $_.processID }
### clean up iedriver and iexplore processes
Get-Process iedriverserver -ErrorAction SilentlyContinue | stop-process
Get-Process iexplore -ErrorAction SilentlyContinue | stop-process
### Get Engine started
Set-Location $EngineLocation
Start-Process -WorkingDirectory $EngineLocation -FilePath "$($EngineLocation)\bin\startEngineManager.bat"
### Get the Desktop session ID
$ts=((qwinsta).Trim() -replace "\s+","," | ConvertFrom-Csv)
$ts=($ts | Where-Object { $_.username -eq "$EngineUser" })
# #$ts[0].id
### Background the RDP session so iedriverserver doesn't fail to keep running
Start-Process -FilePath "C:\Windows\System32\tscon.exe" -ArgumentList "$($ts[0].id)","/dest:console"
$null = Read-Host -Prompt "Press Enter to continue..."
You will also need to edit your startEngineManager.bat to include -Dapmname=ie_syntheticnode so that the PowerShell script can find the node to shut it down during it’s clean-up phase.
startEngineManager.bat (See line 16)
CODE
@echo off
REM -------------------------------------------------
REM Copyright (C) 2014-2019 Germain Software
REM -------------------------------------------------
SET JAVA="%JAVA_HOME%\bin\java.exe"
REM Identify Germain home
IF NOT "%GERMAIN_HOME%" == "" GOTO start
SET GERMAIN_HOME=%cd%
IF EXIST "%GERMAIN_HOME%\bin\apm-engine-manager.jar" GOTO start
cd ..
SET GERMAIN_HOME=%cd%
:start
cd %GERMAIN_HOME%
SET VMARGS=-Dapmname=ie_syntheticnode -Dgermain.jvm=%JAVA% -Xms256m -Xmx512m -XX:+CrashOnOutOfMemoryError
SET JAR=bin/apm-engine-manager.jar
echo Starting Germain Engine Manager...
%JAVA% %VMARGS% -jar %JAR%
You will need to add an argument to your Engine configuration in the Germain Workspace “Germain State” so that the engine can be identified and closed as well by the PowerShell script. -Dapmname=ie_syntheticengine
One Synthetic Scenario at a time
Note:
IE cannot isolate it’s windows from each other. Attempting to run more than one scenario at a time in the same user / session will lead to cross over (cookies, etc…).
The IE driver will have trouble if it’s IE window is not the currently focused window.
Solutions
Option 1: Combine all scenarios into one
You can run all you scenarios from one Germain Component and have them take their turn running. One will run each time the component is called.
If you run the Monitor once per minute and have 15 Scenarios in the Component, each Scenario will be run once every 15 minutes.
Note: All Scenarios need to be shorter than the interval that the Monitor runs on.
We have created a framework to be used in the components code.
See Example Compound Component code here
JAVA
log.info("***** Starting: All IE Synthetic Monitoring *****.");
import org.openqa.selenium.ie.*;
import java.io.File;
import java.lang.StringBuilder;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
//Slot tracking file. Must be writable by user that launched engine.
String rr_path = "C:\\Germain\\GermainEngineIEMonitor\\temp\\rr.txt";
//How many slots to use
int rr_max = 1; //currently uses only the first slot. Set to 7
//**********************************
// Define user variable here (For when the same values are used in multiple slots)
//**********************************
log.info("***** Preparing Creds");
String username="username";
String password="hexstringofpassword";
String host_name = "ServersName";
String host_hostname = "www.hostname.com";
//convert hex in String to characters
StringBuilder output = new StringBuilder();
for (int i = 0; i < kmp.length(); i+=2) {
String str = kmp.substring(i, i+2);
output.append((char)Integer.parseInt(str, 16));
}
password = output;
//Siebel
//Read and update slot tracking file rr_path
int rr_pos;
File f = new File(rr_path);
if(f.exists() && !f.isDirectory()) {
// Read rr file
String t_rr_pos = new String(Files.readAllBytes(Paths.get(rr_path)), StandardCharsets.UTF_8);
rr_pos = Integer.parseInt(t_rr_pos);
rr_pos+=1;
if(rr_pos > rr_max) { rr_pos = 1; }
}else{
//Make it
log.info("***** No rr file '" + rr_path + "'");
rr_pos = 1;
if (f.createNewFile()) {
log.info("***** File created: " + f.getName());
} else {
log.info("***** Could not create '" + rr_path + "'");
}
}
Files.write(Paths.get(rr_path), (""+rr_pos+"").getBytes());
//Temporary Override (If debugging, only run this slot)
//rr_pos = 5;
//Define Slots. You can always add more slots.
log.info("***** Selected: Slot " + rr_pos + " of " + rr_max);
if(rr_pos == 1) {
log.info("***************************");
log.info("***** Slot 1: Running *****");
log.info("***************************");
//This is a rough example of a component that logs into a Knowlege Base, searches the Knowlege Base, then times how long it takes to open a specific decision tree
context.system.name = host_name;
context.system.hostname = host_hostname;
LocalDateTime scriptStartTime = LocalDateTime.now();
org.openqa.selenium.support.ui.WebDriverWait wait = new org.openqa.selenium.support.ui.WebDriverWait(driver, 30);
//Clear Cookies - FAIL Does not clear session
log.info("*** Clearing Cookies");
driver.manage().deleteAllCookies();
//Try to see if we can use a runtime - FAIL Does not clear session
log.info("***** Clearing Cookies via InetCpl...");
Runtime.getRuntime()exec("RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 255");
// Get the login page
String base = String.format("https://%s/app/home", host_hostname);
log.info("***** Getting page");
driver.get(base);
//Close Authentication pop-up
log.info("***** Waiting for Authentication Alert");
Alert alert = wait.until(ExpectedConditions.alertIsPresent());
log.info("***** Closing Alert");
alert.dismiss();
// Search for username / password input and fill the inputs
wait.until(org.openqa.selenium.support.ui.ExpectedConditions.visibilityOfElementLocated(By.xpath("//input[@name='username']")));
log.info("***** Entering Username");
driver.findElement(By.xpath("//input[@name='username']")).sendKeys(kmu);
log.info("***** Entering Password");
driver.findElement(By.xpath("//input[@name='pass']")).sendKeys(kmp);
log.info("***** Clicking");
driver.findElement(By.xpath("//a[@title='Login']")).click();
// Once search page has loaded, wait to let scripts fully execute
wait.until(org.openqa.selenium.support.ui.ExpectedConditions.visibilityOfElementLocated(By.xpath("//input[@name='searchfield_element_name']")));
// Perform search
driver.findElement(By.xpath("//input[@name='searchfield_element_name']")).sendKeys("Search For This");
driver.findElement(By.xpath("//button[@class='SubmitButton']")).click();
// Wait for results
wait.until(org.openqa.selenium.support.ui.ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@class='SearchResultTitleAnswer']")));
//Start Timer
LocalDateTime startTime = LocalDateTime.now();
// Click on result
driver.findElement(By.linkText("Target Link Text")).click();
//Wait for page to load scripted elements
wait.until(org.openqa.selenium.support.ui.ExpectedConditions.visibilityOfElementLocated(By.xpath("//select[@id='elementIdInDecisionTree']")));
// Create login transaction
GenericTransaction tx = new GenericTransaction();
tx.type = "KB:DT Short Name";
tx.name = "DT Long Name";
tx.timestamp = scriptStartTime;
tx.user.name = credentials.username;
tx.duration = DateTimeUtils.computeDuration(startTime, LocalDateTime.now());
tx.success = driver.getPageSource().contains("<select id=\"elementIdInDecisionTree\"");
datamart.insert(tx, context);
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 1: Done ********");
log.info("***************************");
}
if(rr_pos == 2) {
log.info("***************************");
log.info("***** Slot 2: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 2: Done ********");
log.info("***************************");
}
if(rr_pos == 3) {
log.info("***************************");
log.info("***** Slot 3: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 3: Done ********");
log.info("***************************");
}
if(rr_pos == 4) {
log.info("***************************");
log.info("***** Slot 4: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 4: Done ********");
log.info("***************************");
}
if(rr_pos == 5) {
log.info("***************************");
log.info("***** Slot 5: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 5: Done ********");
log.info("***************************");
}
if(rr_pos == 6) {
log.info("***************************");
log.info("***** Slot 6: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 6: Done ********");
log.info("***************************");
}
if(rr_pos == 7) {
log.info("***************************");
log.info("***** Slot 7: Running *****");
log.info("***************************");
//Define Target System in context so submitted fact will contain correct details
context.system.name = "Example";
context.system.hostname = "www.example.com";
//Your Synthetic Monitoring Code here
//Close driver when done
driver.close();
//If you would like to see the details of the fact being submitted
//log.info("Synthetic tx: {}", tx);
log.info("***************************");
log.info("***** Slot 7: Done ********");
log.info("***************************");
}
log.info("***** Exiting: All IE Synthetic Monitoring *****.");
Note:
context.system.namemust be set for each slot in the Component script.
context.system.hostnamemust be set for each slot in the Component script.
Note: In the script above there is a variable tt_path defined. This is the path to a text file that is used to keep track of which slot to use next when the Monitor runes. This file must be writable by the User used to launch the Engine.
Note: When writing Synthetic Monitoring Components for IE there are many Alert Pop-Ups you will need to dismiss. This code snippet can handle them. log.info is optional.
You can avoid scenario / windows focus conflicts by having a separate engine on a separate Windows User or host system (for the Engine) for each Scenario.
If you are on a separate user or system, there will be no cross talk between IE instances, and there will be no focus problems.
Note: This can require a lot of resources
On the Component, must set driver option for Clean Session
If the option isn’t added to the Component, the session settings and cookies will persist between sessions and cause unpredictable and uncontrollable behavior.
To prevent this, add ie.ensureCleanSession=true
Handle "Enhanced Security Configuration"
If IE is set to “Enhanced Security Configuration”, it will add complexity to managing your Synthetic Scenarios.
If you cannot disable it, you will need to clear any alerts that may happen manually(and make sure they do not re-occur).
If you get these alert, check the box to not show them again.
You will also need to Manually perform the scenario and add all involved domains/URLs to Trusted Sites.
If you see this, click “Add…”. Leave the box checked, or it will be much harder to figure out what to add to trusted sites.
Click Add again. Then close the dialogs.
Internet Options: All Security Zone must have the same “Protected Mode” setting
Open “Internet Options” → “Security”
Go to each Zone
Check up uncheck the “Protected Mode” option.
(All zones must be the same)
Internet Options: Must allow Scripting
Most modern sites cannot function without scripting. Verify that Active Scripting is Enabled in these zones:
Launching/Relaunching the Dedicated Engine for IE Synthetic Monitoring
If the Engine with a Desktop Session is interrupted, but being logged out or from a Windows Server Restart, you will need to launch it again. Here are the details.
For convenience I recommend having a few things available.
A shortcut to launch PowerShell as an Administrator
See Section “Create an Admin PowerShell Shortcut”
A stop script to easily shutdown the engine if you need to make changes on the user. (The IE Driver will get in the way)
