Server Installation Instructions
Dependencies
The dependencies of LLview Server are:
- Cron (
crontab) - SQLite 3
-
Perl (>= 5.0)
Install all required modules in one command:(Note: If you plan to connect to an Oracle DB, also installcpan Archive::Tar Compress::Zlib Config::IniFiles Cwd Data::Dumper DBD::SQLite DBD::SQLite::BundledExtensions DBI Exporter File::Monitor File::Spec FindBin Getopt::Long IO::File JSON LWP::Protocol::https LWP::Simple LWP::UserAgent Parallel::ForkManager POSIX Storable SVG SVG::TT::Graph::Line Text::CSV_XS Time::HiRes Time::Local Time::Zone Tk Tk::NoteBook Tk::Table warnings::unused YAML::XSRPC::PlClient.) -
Python (>= 3.9) (used by JuRepTool and auxiliary API/log/git plugins)
(Note: If compressed HTML reports are to be generated with
Install all required packages in one command:--gzip, Python must have been built withgzipsupport.) - Sans-serif: Liberation Sans or Arial
- Monospace: Liberation Mono or Courier New
(See JuRepTool Font Installation below for setup instructions.)
Configuration Overview
The .llview_server_rc File
The primary environment configuration file for LLview Server is ~/.llview_server_rc. It defines environment variables used across the server workflows:
$LLVIEW_SYSTEMNAME: Unique identifier for the monitored HPC cluster.$LLVIEW_HOME: Path where the corellviewrepository was cloned.$LLVIEW_CONF: Path to your active configuration folder (outside the git repo).$LLVIEW_DATA: Storage folder for databases, temporary files, and logs. To use dedicated storage, either mount it directly here or link it:$LLVIEW_SHARED: Shared folder between LLview Remote and LLview Server where files are staged. Must match$LLVIEW_SHAREDin.llview_remote_rc.$LLVIEW_SHUTDOWN: Path to a trigger file used to pause execution without modifying cron (managed automatically viallview startandllview stop).$LLVIEW_LOG_DAYS: Number of days to retain server logs.$LLVIEW_CONF_FILE: Path to.llview_server_rcitself (allows the daemon to detect changes).$PYTHON: Absolute path to the Python interpreter satisfying the dependencies.$JUREPTOOL_NPROCS: Worker processes for JuRepTool. Recommendation: Setexport JUREPTOOL_NPROCS=0during initial setup, and only enable it once the main LLview cycle is verified.$LLVIEW_WEB_DATA: Destination folder on the Web Server where generated files will be copied.$LLVIEW_WEB_IMAGE: Path to the system image on the login page (default:img/$LLVIEW_SYSTEMNAME.jpg).
Installation Steps
1. Clone the Code Repository
Clone the LLview code repository to your designated $LLVIEW_HOME:
2. Set Up Configurations ($LLVIEW_CONF)
Configuration templates are maintained in the llview-config repository.
Important: Do not work directly inside the cloned
llview-configrepository. Working in a separate folder allows you to rungit pullon the repo in the future to receive upstream updates cleanly.
-
Clone the configuration template repository:
-
Deploy your configuration directory:
- Recommended (Inheritance method): Copy
example_system/out of the repository into your own directory (e.g.,~/my_cluster_config). This path becomes your$LLVIEW_CONF. Becauseexample_system/server/LLgenDB/LLgenDB.yamlreferencescommon/server/LLgenDB/LLgenDB_common.yamlinside the repo, your installation automatically inherits upstream updates whenever yougit pullthe repo. -
Standalone method: Copy the entire
common/folder out of the repository to your target directory and modify all files locally. -
Deploy
.llview_server_rc: Copy$LLVIEW_CONF/server/.llview_server_rcto your home folder (~/.llview_server_rc) and customize the paths. -
Source your environment:
3. Customize Actions and Permissions
LLview Server processes data using actions (first-level workflows) made up of steps (second-level tasks).
- Edit
$LLVIEW_CONF/server/workflows/actions.inpto enable required actions.Tip: Start with all actions set to
active=0and activate them one by one while checking log files. - Edit individual action files under
$LLVIEW_CONF/server/workflows/(e.g.,LML_da_collect.conf,LML_da_dbupdate.conf). - Set up the webservice step under the
collectaction to map project and user permissions. - Set up the transfer key pair for sending data to the Web Server.
4. Initialize Database Tables
Before launching the server, SQLite databases and tables must be initialized from your YAML configurations in $LLVIEW_CONF/server/LLgenDB/:
Reminder: Run
updatedbwhenever you make changes to YAML configuration files inLLgenDB/.
5. Schedule the Server Daemon
LLview Server uses a monitor daemon launched periodically via cron to ensure high availability. If the daemon is already running, the cron script simply exits.
Append the cron entry safely:
Verify the entry:
crontab -l
# Expected output:
* * * * * . ~/.llview_server_rc ; perl "$LLVIEW_HOME/da/workflows/server/crontab/serverAll.pl"
Detailed Workflow Actions and Modules
collect Action
The collect action runs first. It retrieves raw data generated by Remote from $LLVIEW_SHARED and runs auxiliary server-side data collection plugins to assemble the LML files.
webservicestep:
This is a critical step to configure for each installation.
To establish role-based access in LLview (restricting users to their own jobs/projects, mentors to mentored projects, and support to all jobs), user and account mapping data is required.
In the example configuration, this is generated every 15th cycle via$LLVIEW_HOME/da/utils/exec_every_n_step_or_empty.plrunning an account script (such as JSC's internalget_webservice_accounts.pl). The output is written to$LLVIEW_DATA/$LLVIEW_SYSTEMNAME/perm/wservice/accountmap.xml. LLview uses this file to build folder structures and.htaccessaccess lists.
See Account Mapping Documentation for details on creating this file. Support users can also be supplied via thesupportinputaction.
dbupdate Action
The dbupdate action ingests the collected LML files into SQLite databases and triggers post-processing workflows.
LMLDBupdatestep: Inserts processed LML records into SQLite tables according to definitions in$LLVIEW_CONF/server/LLgenDB/. (Requires tables to be initialized viaupdatedb).trigger_JobRepstep: Signals thejobreportaction to generate web reports in parallel.combineLML_allstep: Merges individual LML snapshots into a single archive file (used for historical review and replay).
jobreport Action
Creates web data and exports it to the Web Server according to definitions in $LLVIEW_CONF/server/LLgenDB/.
transferreportsstep:
Transfers data securely from the LLview Server to the Web Server using SSH andrrsync.- Generate a dedicated SSH key pair without a passphrase on the LLview Server:
- On the Web Server, append the public key to
~/.ssh/authorized_keyswith restricted options:(Note:from="<LLVIEW_SERVER_IP>",command="<path_to_rrsync>/rrsync.pl -wo <destination_data_folder>",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding <public_key_string>rrsync.plis bundled with JURI under$JURI_HOME/utils/). - Update the step parameters with your Web Server SSH settings:
- Perform an initial connection from the LLview Server to accept the host key:
icmap Action (Interconnect Mapping)
Used to color compute nodes in detailed reports based on network rack or cell topology.
- Provide node-to-cell mappings in an icmap file located in ${LLVIEW_DATA}/${LLVIEW_SYSTEMNAME}/perm/icmap:
icmap file is available in the llview-config repository under server/).
- When updated, it triggers $LLVIEW_HOME/da/utils/get_hostnodemap.py to create icnodemap.xml, which is then imported into the database by dbupdate.
supportinput Action
Watches a user file (default: ${LLVIEW_SHARED}/../config/support_input.dat) containing support usernames (one per line). When updated, it syncs the list to ${LLVIEW_DATA}/${LLVIEW_SYSTEMNAME}/perm/wservice to assign portal-wide support permissions.
Maintenance Actions (compress, archive, delete)
Manages historical temporary files in ${LLVIEW_DATA}/${LLVIEW_SYSTEMNAME}/tmp/jobreport/tmp/mngtactions to prevent excessive disk space consumption.
JuRepTool (PDF and HTML Job Reports)
JuRepTool generates standalone job metric reports when triggered by ${LLVIEW_SYSTEMNAME}/tmp/plotlists.dat.
- Configuration: Edit YAML files under
$LLVIEW_CONF/jureptool/: config.yml: Visual styling, page size, timezone, and hostname.system_info.yml: Cluster topology, core counts, memory sizes, and queue definitions.plots.yml: Plot sections and graph definitions.logging.yml: Logging destinations and verbosity.- Install Fonts:
Install Liberation or Arial/Courier fonts (see Dependencies): - Enable Processing: Set
$JUREPTOOL_NPROCSto a value greater than0in~/.llview_server_rc.
Service Control and Monitoring
Service Commands (llview)
You can control LLview Server execution using the built-in control commands:
- Pause processing:
(This creates
$LLVIEW_SHUTDOWN. The cron task continues to trigger each minute, but immediately halts without starting actions). - Resume processing:
(This removes
$LLVIEW_SHUTDOWNand allows the daemon to resume processing). - Stop permanently: Kill the daemon and remove the cron job: