Agenda

1. Documentation and help
2. JavaScript as automation language
3. Apps, fields, flows — Code organisation, Groups
4. Modules — Desktop and OS, User interaction, Documents, External modules
5. Other configuration options — Tables, Secrets, Restrictions, Services
6. Practical tips

Slides are available at http://slides.sirenia.io/developer-2

Documentation

The info given today (and more) is also available on https://docs.sirenia.io. Default docs version is 2.0, however you should go to the 2.1 version of you have that version of Manatee installed.

Forum

We are also active on https://ask.sirenia.io where you can ask questions and get answers. It also mirrors the documentation site s.t. you can search and find lots of useful information directly here.

These slides are also available on https://slides.sirenia.io/developer-2.

JavaScript

Features

• Simple syntax, easy to learn
• Dynamic typing
• First-class functions
• Object-oriented programming
• Single-threaded
• ECMAScript standard (current: ES2023, we support ES2015)
• One of the core technologies of the web alongside HTML and CSS

JavaScript

Hello, World!

// Basic output - I am comment btw
Dialog.info("I'm a dialog"); // alert("I'm a popup!");
Log.info("", "I appear in logs"); // console.log("I appear in the console");

/* Variables
 * I'm another comment, multiline */
let name = "JavaScript";
const version = "ES2015";
var oldWay = "Not recommended";

JavaScript

Data types

// Primitive types
let string = "Text";
let number = 42;
let boolean = true;
let nullValue = null;
let undefinedValue;

// Object type
let object = { name: "JavaScript", year: 1995 };

// Array type
let array = [1, 2, 3];

JavaScript

Control Flow

// Conditionals
if (condition) {
  // code
} else if (anotherCondition) {
  // code
} else {
  // code
}

// Switch statement
switch (value) {
  case 1:
    // code
    break;
  default:
  // code
}

// Loops
for (let i = 0; i < 5; i++) {
  Log.info("", i);
}

while (condition) {
  // code
}

JavaScript

Functions

// Function declaration
function add(a, b) {
  return a + b;
}

// Function expression
const multiply = function (a, b) {
  return a * b;
};

// Arrow function
const divide = (a, b) => a / b;

// Default parameters
function greet(name = "User") {
  return `Hello, ${name}!`;
}

JavaScript

Arrays and Objects

// Arrays
let fruits = ["Apple", "Banana", "Cherry"];
fruits.push("Date");
fruits.pop();
fruits.forEach((fruit) => Log.info(fruit));

// Objects
let person = {
  name: "John",
  age: 30,
  greet: function () {
    return `Hello, I'm ${this.name}`;
  },
};

JavaScript

Exception handling

const badFunc = () => throw "Bad function";

try {
  badFunc();
} catch (e) {
  // We're safe here
  Log.warn("", e + " happened.");
}

Fields

• UI elements that can be interacted with programmatically
• Represent buttons, text inputs, dropdowns, tables, etc.
• Identified by a path (preferred) or screenshot
• Fundamental building blocks for UI automation

// Interacting with fields
// submitButton is already defined
Fields.submitButton.click();
let userName = Fields.nameInput.read();

// dynamically declared
let passwordField = Fields.passwordField;
passwordField.input("my-secure-password");

Fields

Paths

Field paths identify UI elements by providing a “route” to field. They can be fairly complex:

Fields

Operations

// Basic interactions
Fields.someButton.click();
Fields.someTextField.input("Hello World");
Fields.someDropdown.select("Option 2");

// Get information
let properties = Fields.element.inspect();
let isVisible = Fields.element.exists();

// Advanced
Fields.container.findAll("**/Button").forEach((button) => {
  Log.info(`Found button: ${button.read()}`);
});

For more info see field documentation.

Flows

Represents an (automation) action or context state for a specific app.

Action

A series of steps to be executed within the application. Does not necessarily have to interact with the application itself, but can.

Context State

Steps to take to read state from an application or affect the application to change its state to match the shared context.

Flows

Code organisation

• Share code between flows with Flow.include. MODULE flows.
• Run other flows with Flow.run('other flow').
  • If the flow to run is associated with another app, then Manatee will attempt to launch the app, if it is not already running.
  • A special type of app is headless app which does not need to be launched and is not associated with an actual application.
• Flows may have inputs and outputs.

Flows

Groups

Groups are used to define who has access to a given app or flow. The rule is that you need at least two groups; the group that the Manatee instance you run is configured with – we call this the primary group – and then another group which should match your username, machine-name or an AD-group that you are a member of.

An example; if you are running an instance of Manatee which is configured to use PRODUCTION as its primary group and your username is johndoe then you need at least two groups:

• PRODUCTION and
• johndoe

Flows

Running a flow: Flow menu

Users may manually trigger a flow vhen you hold down the alt key and right-click in the attached app.

Other factors;

• groups,
• anchors, and
• defined intpus.

Flows

Running a flow: Triggers

Custom conditions for when flows should run.

Some trigger based on user actions, others on changes in the “environment”.

Flows

Context

Desktop and OS

• HID to use low-level mouse and keyboard input
• Fs filesystem operations
• Windows interact with desktop windows
• Clipboard read and write to the clipboard
• Env access to environment variables

Desktop and OS

Human Interface Devices (HID)

• Mouse to move, click

// All mouse functions are chainable meaning you can move, then click.
Mouse.moveTo(10, 10).click();

• Keyboard to type

// The long way to an "F"
Keyboard.down(Keyboard.SHIFT).press("f").up(Keyboard.SHIFT);

// Send 2 TAB keys followed by 'f', 'o', 'o' and then `ctrl+a` to select all.
Keyboard.send("{TAB 2}foo^a");

Desktop and OS

Filesystem

• Fs interacts with the local file system
• Key operations:
  • List files/directories: Fs.ls('C:\\folder\\*.txt')
  • Create directories: Fs.mkdir('C:\\folder')
  • File operations: Fs.read(), Fs.write(), Fs.cp(), Fs.mv(), Fs.rm()
  • Check existence: Fs.exists('C:\\folder\\file.txt')

// Read and write files
let content = Fs.read("C:\\folder\\file.txt");
Fs.write("C:\\folder\\output.html", "<h1>Hello World</h1>");

// File management
Fs.cp("C:\\source.txt", "C:\\dest.txt", { overwrite: true });

Desktop and OS

Windows Module

Inspect and manipulate desktop windows.

• Find all desktop windows - Windows.all()
• All application windows - Windows.forApp()
• Main application window - Windows.primary

// Window manipulation
let mainWindow = Windows.primary;
mainWindow.move(100, 100);
mainWindow.resize(800, 600);
mainWindow.maximize();

// Screenshot and keyboard input
let screenshot = mainWindow.screenshot();
mainWindow.sendKeys("Hello World");

Desktop and OS

Clipboard

• .get(), .set(), .clear(), .copy(), .cut() and .paste()

Env

Read and in some cases write:

• user and machine info,
• printers
• and lots of other environmental info

User Interaction

• Dialogs
• Notifications
• Stickies
• Waiting for things to happen
• Tip, mask and follow

User Interaction

Dialogs

// Simple info and warning dialogs
Dialog.info("Info", "This is an informational message");
Dialog.warn("Warning", "This is a warning message");

// Input dialog with multiple field types
let result = Dialog.input("User Input", "Please provide information", {
  name: { type: "TEXT", prompt: "Your name", validation: { isRequired: true } },
  age: { type: "NUMERIC", prompt: "Your age", min: 0, max: 120 },
  role: {
    type: "SELECT",
    prompt: "Your role",
    selectBetween: ["Admin", "User", "Guest"],
  },
});

Validation is also possible. Basic interaction; when certain value is selected show some input.

User Interaction

Notifications

Simple, non-interactive popups for user awareness.

// Basic notification types
Notification.show("process", "Process ongoing", "Please wait ...");
Notification.update("process", "Process complete", "All done!", { timeout: 5 });

// With callback when clicked
Notification.show("Info", "Click me", 10, function () {
  // Do something in case of click
});

User Interaction

Stickies

// Basic sticky with actions
Sticky.open("mySticky", {
  title: "My Sticky",
  items: [
    {
      type: "MARKDOWN",
      text: "## Sticky Example\nThis is a *persistent* window",
    },
    {
      type: "ACTION",
      name: "runReport",
      header: "Run Report",
      body: "Click to generate report",
    },
  ],
});

User Interaction

Stickies

// Progress tracking in sticky
let mainProgress = new Progress("Main Task", [
  new Progress("Step 1"),
  new Progress("Step 2"),
  new Progress("Step 3"),
]);

Sticky.open("progressSticky", {
  title: "Task Progress",
  items: [{ type: "PROGRESS", progress: mainProgress }],
});

mainProgress.next(); // Completes Step 1, starts Step 2

User Interaction

Tips and Masks

Highlight UI elements and guide users.

// Show a tip attached to a UI element
Fields.middleButton.tip({
  text: "I am tip",
  pointing: "left",
});

// Mask a UI element to prevent interaction
let mask = Fields.dangerousButton.mask({ color: "Red" });
// Later remove the mask
mask.close();

User Interaction

Wait Module

Control flow execution timing.

// Wait for specific durations
Wait.forSeconds(2);
Wait.forMilliseconds(500);

// Wait for UI elements
Wait.forField(Fields.resultTable, 10); // Wait up to 10 seconds for field to appear
Wait.forFieldToDisappear(Fields.loadingSpinner, 5);

// Wait for window to appear/disappear
Wait.forWindow("Report Generator", 10);
Wait.forWindowToDisappear("Loading...", 5);

// Wait for user interaction
Wait.forClick(Fields.confirmButton, { timeout: 30000 });

// Wait for custom conditions
Wait.for(
  function () {
    return Fields.statusText.read() === "Complete";
  },
  { timeout: 15000, interval: 500 },
);

Documents

Word Module

Inspect and analyze Word documents with a DOM-like structure.

// Load the Word module (using versioning)
var word = Module.load("Word", { version: "2.2" });

// Inspect a document to access its structure
var doc = word.open("C:\\Documents\\report.docx");

// Access document structure and convert to HTML
var htmlContent = doc.Children[0].toHtml();

// Both get and set of .text
var secondChildText = doc.Children[2].text;

// Search and replace
doc.replace(/find/g, function (match) {
  return match.toUpperCase();
});

Documents

Excel Module

Powerful spreadsheet manipulation capabilities.

// Load entire spreadsheet with headers
var table = Excel.load("spreadsheet.xlsx", {
  table: { range: "A1:D10", worksheet: "Sheet1", header: true },
});

// Access data by column name (when header: true)
for (var i = 0; i < table.length; i++) {
  Log.info("Customer: " + table[i].CustomerName + ", Value: " + table[i].Value);
}

// Update single cell
Excel.updateCell("data.xlsx", "Sheet1", "A3", 1000);

// Update multiple cells at once
Excel.updateCells("data.xlsx", "Sheet1", "A1", [
  [10, 20, 30],
  [40, 50, 60],
]);

// Delete operations
Excel.deleteRows("data.xlsx", "Sheet1", [3, 4, 5]);

Documents

PDF Modules

Extract text and create PDFs. Two modules.

// Extract text blocks from PDF
var pdf = Module.load("Pdf", { version: "1.0" });
var result = pdf.textBlocks("/path/to/file.pdf", { page: 2 });

// Find text relative to other elements
var block = result.find("below", "Invoice Total");
Log.info("Amount: " + block.text);

// Create PDF from HTML
var pdfBuilder = Module.load("PdfBuilder", { version: "2.2" });
var doc = pdfBuilder.create("<h1>Report</h1><p>Generated content</p>", {
  paperSize: "A4",
  paperOrientation: "Landscape",
});
doc.saveAs("/path/to/output.pdf");

// Work with PDF forms
var formDoc = pdfBuilder.open("/path/form.pdf");
formDoc.writeForm({ name: "Jane Doe", email: "jane@example.com" });

Database Access

Db Module

Powerful database connectivity for automation. SQLite, MSSQL, Oracle, PostgreSQL, MS Access

// Connect to a database
var db = Db.connect("sqlite", "Data Source=C:\\Data\\foo.db");

// Query data
var products = db.query(
  "SELECT id, name, price FROM Products WHERE price > @price",
  {
    "@price": 100,
  },
);

// Use results
for (var i = 0; i < products.length; i++) {
  Log.info("Product: " + products[i].name);
}

// Execute commands
db.exec("CREATE TABLE Customers (id INT, name TEXT, email TEXT)");
db.exec("INSERT INTO Customers VALUES (@id, @name, @email)", {
  "@id": 1,
  "@name": "John Smith",
  "@email": "john@example.com",
});

Database Access

Transactions

Ensure data consistency with transaction support.

// Begin a transaction
var tx = db.begin();

// Execute multiple statements as one unit
tx.exec("INSERT INTO Orders VALUES (@id, @date, @customer)", {
  "@id": 1001,
  "@date": new Date(),
  "@customer": 42,
});

// Commit the transaction if everything succeeded
tx.commit();

External Modules

We have a bunch of external modules which can be loaded dynamically via Module.load(<name>, <version>).

A selection:

• Audio — speech synthesis and audio playback
• Compress — zip, unzip
• Html — parse and manipulate HTML
• Http, Ftp — network clients
• Image — image manipulation
• Network — Swiss-army knife of networking
• Task — asynchronous tasks

Configuration

Services

Shared connections to mail, chat, serial comms and message queues.

Has both triggers and flow interaction via Service module.

• Mail — send, list, read and delete mails
• Serial comms — send and receive data
• Message queue — send and receive messages
• Chat — send and receive messages

Configuration

Tables

Tables is a light-weight built-in database. UI in Cuesta and Table module for access in Manatee.

Configuration

Secrets

Encrypted storage for confidential information.

Secrets module can be used to access secrets if allowed.

Configuration

Restrictions

Control access to API or add arbitrary rules to flows.