2.1   Discovering available configurations¶

The easiest way to get information from a CSC programmatically is by using a Jupyter notebook server. From a notebook, observers, developers and power users can easily interact with the system through Python.

In order to know which configurations are available for a specific CSC, users can read the configurationsAvailable event. This is done using a salobj.Remote class.

from lsst.ts import salobj

domain = salobj.Domain()

atdome = salobj.Remote(domain, "ATDome")

await atdome.start_task()

config_available = await atdome.evt_configurationsAvailable.aget(timeout=5.)

# This will print the available labels.
print(config_available.labels)

# This will print the mapping between label and configuration files.
print(config_available.mapping)

Often, this can also be accomplished using a high-level class that is designed to interact with a group of CSCs.

from lsst.ts.observatory.control import ATCS

atcs = ATCS()

await atcs.start_task

config_available = await atcs.rem.atdome.evt_configurationsAvailable.aget(timeout=5.)

# This will print the available labels.
print(config_available.labels)

# This will print the mapping between label and configuration files.
print(config_available.mapping)

It is also possible to check this information by querying the EFD or through the CSC summary information interface on LOVE. Examples of how to do this using the LOVE interface will be added when the functionality is ready.

2.2   Selecting default configuration¶

In most cases, the control packages contain high-level commands to enable all components under their control, and select the default configuration in the process. An example of this is the ATCS.

from lsst.ts.observatory.control import ATCS

atcs = ATCS()

await atcs.start_task

await atcs.enable()

If working with an individual CSC, which as an operator would be a rare occurrence, default CSC configurations are loaded simply by transitioning the CSC via:

from lsst.ts import salobj

domain = salobj.Domain()

atdome = salobj.Remote(domain, "ATDome")

await atdome.start_task()

# CSC needs to be in STANDBY state for this to work
await salobj.set_summary_state(atdome, salobj.State.ENABLED)

2.3   Selecting a non-default configuration¶

Selecting non-default configurations via control packages is also possible. A dictionary is used to override the appropriate configuration labels for each component that needs a non-default configuration. This example assumes the component of interest is already in the STANDBY state.

from lsst.ts.observatory.control import ATCS

atcs = ATCS()

await atcs.start_task

# ATAOS must be in STANDBY state for this to work. All other CSCs will
# be configured with the default label
await atcs.enable(configuration={'ATAOS': 'constant_hex'})

Examples of how to do this using the LOVE interface will be added when the functionality is ready.

If working with an individual CSC, which as an operator would be a rare occurrence, the salobj.Remotes class may be more appropriate:

from lsst.ts import salobj

d = salobj.Domain()

atdome = salobj.Remote(d, "ATDome")

await atdome.start_task()

await salobj.set_summary_state(
atdome, salobj.State.ENABLED, configurationToApply="original-install"
)

2.4   Changing the default configuration¶

Changing the default configuration is a more involved endeavor because it entails making a change to the contents of the configuration repository. Because the repo is under version control, the appropriate steps must be taken. For this example, let’s assume we want to change the default configuration in the ATAOS, which is found in the ATAOS directory of the ts_config_attcs repo.

1. Create a JIRA ticket in where the title/description note the change being made. Let’s assume it creates ticket DM-12345.

2. Clone the repo and checkout a new branch

   git clone git@github.com:lsst-ts/ts_config_attcs.git
   git checkout -b tickets/DM-12345
   

   Note that the branch name is the word tickets/ appended with the Jira ticket name.

3. Open the most recent schema version (v2) and modify the contents of _labels.yaml. For example, the original version may be:

   # Labels for recommended configurations; a dictionary of {label: config_file}
   default: _base.yaml
   constant_hex: hex_m1_202003_constant_hex.yaml
   

   Say you wish to add a new configuration label called, m1_hex, and then make the constant_hex be the default. Because _base.yaml is always assigned the label of default, the contents of _base.yaml must be updated to include the contents of constant_hex. Then, m1_hex and the corresponding filename needs to be appended. Therefore, the file would become:

   # Labels for recommended configurations; a dictionary of {label: config_file}
   default: _base.yaml
   hex_m1: hex_m1_hex_202003.yaml
   

4. Add, commit and push the changes, with a commit message.

   git commit -am "Updated base configuration to use the configuration previously labeled constants_hex with a filename of hex_m1_hex_202003.yaml. Added hex_m1_hex_202003.yaml with label of hex_ml. See DM-12345 for more information."
   git push
   

   The commit message can add information about what changes are being made and a short description for the reason. It is also recommended to explicitly mention the Jira ticket for the work being done as the branch name is lost once the changes are merged to the head branch.

5. If this is a normal configuration change procedure, then create a pull-request (PR), and have it reviewed, merged and released.

6. Once the new configuration is released it can be made available to the component, which will not automatically see the newly created configuration. During normal operations this involves creating a new deployable artifact and updating the deployment to use the new configuration version.

   On-the-fly changes are discouraged but sometimes a reality and are therefore discussed in 3   Creating a new configuration.

7. Once the component is re-deployed with the new configuration, bring it from STANDBY to the ENABLED state. No explicit specification of the configuration is necessary since the default is being selected. If a different label is used, the configuration parameter must be set in the command below (see 2.3   Selecting a non-default configuration).

   await salobj.set_summary_state(ataos, salobj.State.ENABLED)
   

2.5   Finding a previously used configuration¶

In the future, one may want to verify which configuration was being used for a given observation. Because we often use generic labels (e.g. default), and file contents can change with time, creating a robust version controlled system must go beyond simply changing filenames. For this reason, additional metadata is associated with each configuration, notably the url and version parameters in both the configurationsAvailable and configurationApplied events. These parameters are key to ensuring that each configuration is unique, and is traceable to their filename and contents.

The url parameter simply contains a URL indicating how the CSC connects to its settings (meaning a link to the repository). The version parameter is more complicated. For all CSCs (except the camera?), the version parameter is a branch description[1] which is automatically generated and populated by the CSCs. This is what is output by running the following command in a configuration repository (e.g. ts_config_latiss):

git describe --all --long --always --dirty --broken

An example output is, heads/develop-0-gc89ef1a. The repository branch (or tag) name forms the first part of the branch description. This first part contain individual identifiers and can change rapidly. It may take any form necessary to convey the appropriate information. The last 7 characters (c89ef1a) is the hash of the commit of the loaded configuration file. Users can find this commit by navigating to the repository on github, searching for the commit hash, then clicking on the “commits” section of the search results, as shown in the screenshot below.

Figure 1 Using the version output in the configurationApplied event, it is possible to traceback the repo to the configuration that was loaded.

2.6   Exceptions¶

TBR.

3.1   On-the-fly changes¶

During the process of creating a new configuration (3   Creating a new configuration) or during a commissioning/engineering run, it may be necessary to make a new configuration available to a running CSC for testing without rebuilding/re-deploying the component. In these cases, the user should also create a Jira ticket (or work out of an existing ticket) to document the occurrence.

Following are the steps to make a new configuration available to a running CSC:

1. If the configuration is not already created and pushed to GitHub, follow steps 1 to 5 in 3   Creating a new configuration.

2. Make sure the CSC in in STANDBY state, which can be accomplished using the following command.

   await salobj.set_summary_state(ataos, salobj.State.STANDBY)
   

3. Login to the where the CSC is running. The procedure will vary depending on how the CSC is deployed. Most Telescope and Site components are deployed on containers using Kubernetes (k8s). For CSCs that are not running on a container, you should be able to login to the host machine with ssh and continue with the procedure (go to step 3). A provisory list of IPs can be found in confluence <https://confluence.lsstcorp.org/x/qw6SBg>. For details about the deployment system see the deployment documentation.

   The procedure to access containerized components is as follows:

   1. Log in to the rancher service at https://rancher.ls.lsst.org. You will need special authorization to acquire an account on that service.

      Warning

      This service is responsible for managing the deployment of the entire system. Make sure you follow the procedure exactly. If you are in doubt about an operation make sure you verify it with knowledgeable personnel.

   2. Once logged in, you will be presented with the list of available k8s clusters.

      

      Figure 2 List of Kubernetes clusters. At the time of this writing, the only cluster available was kueyen, the commissioning cluster at the base facility in Chile.

      Click on the name of the cluster where the CSC you want to modify is running. If it is a summit operation, the name of the cluster will be andes. After selecting the cluster, you will be redirected to the cluster dashboard.

      

      Figure 3 Cluster dashboard.

   3. On the top right corner of the cluster dashboard, there is a button with Launch kubectl. This will open an interactive session on you browser that will allow you to interact with the k8s cluster you selected. If you are knowledgeable about k8s you can also download the Kubeconfig file and login to the cluster from your own computer.

      Warning

      Do not download the Kubeconfig file unless you really know what you are doing. This file contains access and credential information that would allow users direct access to the k8s cluster.

   4. Once you select Launch kubectl you will be redirected to a Shell connected directly to the selected k8s cluster.

      

      Figure 4 Kubectl shell.

   5. Use the following command to discover the container running the CSC :

      kubectl get pods -n cscs
      

      This will list all the CSCs “pods” which are, basically, the running containers. The name of the CSC will be part of the pod name and should be easy to identify.

   6. Connect to the running pod:

      kubectl exec -it -n cscs <pod-name> -- /bin/bash
      

      Make sure to replace <pod-name> with the name of the pod for that CSC.

4. Once inside the CSC host, go to the location where the configuration is installed. This information can be found in the CSC documentation or in the deployment documentation. You should be able to use regular linux command line commands (e.g. ls and cd).

5. Once in the configuration package, update the git repository and checkout the branch with the new configuration:

   git fetch --all
   git checkout -b tickets/DM-12345
   

6. Once the branch is updated you can re-enable the component to load the new configuration.

   await salobj.set_summary_state(ataos, salobj.State.ENABLED)
   

The version attribute in the configurationsAvailable event would reflect that change with something like:

version: heads/tickets/DM-12345-0-g79e2257

Note that it would be possible to track the configuration in the future, even if the branch is removed from the repository, by using the commit hash (g79e2257).

3.2   In-line changes¶

During commissioning, we anticipate that there will be situations where quick configuration changes need to be implemented and tested. In these cases, working out of a local branch and going over the 3.1   On-the-fly changes process may result in the loss of on-sky time. To ensure the work/changes is tracked it is still recommended that the user create a Jira ticket (or work out of an existing ticket) to document the occurrence. Then, instead of checking out the repository locally, the user can work out of the deployed CSC configuration directly in the host.

To do this, perform the following procedure:

1. Verify (or transition) the CSC in in STANDBY state.

   await salobj.set_summary_state(ataos, salobj.State.STANDBY)
   

2. Login to the where the CSC is running. The procedure will vary depending on how the CSC is deployed. For containerized components, you can find details on how to do that in the deployment documentation.

3. Once inside the CSC host, go to the location where the configuration is installed. This information can be found in the CSC documentation or in the deployment documentation.

4. Create a local branch to work on.

   git checkout tickets/DM-12345
   

5. Use the available text editors (vim and emacs are usually made available) to edit the configurations.

6. Once the configurations are edited and saved, re-enable the component.

   await salobj.set_summary_state(ataos, salobj.State.ENABLED)
   

It is important to create a branch in place to work on and, later, commit-push to the repository and continue with the process afterwards.

Transient labels with Jira ticket numbers may be used for developing new configurations. They should be moved to standard type labels at the earliest opportunity.

Note that when you connect to the computer running a CSC and edits the configuration directly, the version parameter reflect that change with something like:

version: heads/tickets/DM-12345-0-g79e2257-dirty

When this happens, it prevents us from precisely identifying what configuration was used. In this case, the preferred solution is to use 3.1   On-the-fly changes to ensure traceability is not lost, at the expense of a couple extra minutes.

3.3   Exceptions¶

The following require different procedures to create/modify a configuration

• Main and Auxiliary Telescope Pointing Components
• M2
• ATMCS and ATPneumatics

7.1   Pointing Component¶

The pointing component has a configuration file that resides with the code base which, in itself, also defines a couple different files (e.g. pointing model). Nevertheless, the CSC is not developed to be a configurable CSC, meaning it does not accept a configurationToApply value to switch between different configurations and does not output the required events.

The CSC is being developed by Observatory Sciences using C++.

7.2   M2¶

7.3   ATMCS and ATPneumatics¶

7.4   Non-Configurable CSCs¶

Some CSCs will not be configurable at all. Examples are sparse in our current architecture but, the from Salobj point of view, a CSC can be developed on top of a BaseCSC which makes it a non-configurable component.

A non-configurable CSC will ignore the configurationToApply attribute of the start command, as it does not contain any true meaning to it. Likewise these CSCs will not output any of the configuration-related events.

References