FireWorks

Getting Started

Select FireWorks as the plugin. There are no Application Parameters for this plugin.

FireWorks (browser-based FireWorks IDE) will launch in a browser. Click the ‘Load’ button to get started.

FireWorks Landing Page
FireWorks Landing Page

Let’s create a couple simple firetasks for testing purposes.

  1. Create two files, task 1 and task2
  2. Add ‘hostname’, echo statements inside the firetasks and redirect output.
  3. Make the tasks executable
FireWorks Create Tasks
FireWorks Create Tasks

Now that the firetasks are ready, add the firestasks to the launchpad and create the workflow. The firetasks can be separated and created with individual workflows as well.

  1. lpad add_scripts ~/task1 ~/task2 -n fw_task1 fw_task2 -w fw_rapid_tasks
FireWorks Add Tasks
FireWorks Add Tasks

Workflows can be removed by performing the following:

  1. lpad delete_wflows -i ‘id’

A refresh on the dashboard will show the additional firetasks and a workflow in a ready state. The interface also provides graphs and a database snapshot of current activity while working through the progress.

FireWorks WorkFlow Ready State
FireWorks WorkFlow Ready State

FireWorks supports both singleshot and rapidfire commands. The rapidfire will send a go to each firetask in a ready state; one at a time while the singleshot is a target approach.

Before we can run qlaunch, the ‘my_fworker.yaml’ file will have to be modified, (see notes located at bottom). The workflow name given in the -w argument above during the add firetasks process should be inserted after the ‘name:’ key value pair. In our example above, we should insert ‘name: fw_rapid_tasks’.

FireWorks My FWorker YAML Modification
FireWorks My FWorker YAML Modification

Launch Rocket:

  1. qlaunch -l ~/.fireworks/my_launchpad.yaml -q ~/.fireworks/my_qadapter.yaml -w ~/.fireworks/my_fworker.yaml rapidfire -m 3 -b 1 –nlaunches 0

-m = Max number of jobs
-m should never be then the number of tasks + 1 (the + 1 is the server node)
-b = Number of batches
–nlaunches = Number of qsubs to do before exiting qlaunch

Verify jobs launced with ‘qstat’.

FireWorks Qstat Jobs Verify
FireWorks Qstat Jobs Verify

The dashboard will show the associated workflow is in a running state.

FireWorks WorkFlow Running State
FireWorks WorkFlow Running State

The dashboard will then track when the jobs have been completed.

FireWorks WorkFlow Completed State
FireWorks WorkFlow Completed State

The dashboard also allows for a quick query mechanism. The following will allow for a query targeting a specific identifier.

  1. After the URL domain.com, add /wf/id (workflow identifier). For example: replace https://domain.com with https://domain.com/wf/29
FireWorks WorkFlow Query identifier
FireWorks WorkFlow Query identifier

Notes: Configuration yaml files are generated during each plugin launch. The yaml files contain the default configurations to allow FireWorks to communicate with the PBS Batch Management System. The .fireworks working directory is located in the home directory.

Please work from your home or the .fireworks directories. The following cleanup action will take place during launch of the FireWorks plugin, (FW.json, FW_submit.script files and block_* directories).

External References

For more information on how to use the FireWorks IDE, please visit nersc.gov or materialsproject.github.io