Guide version 1.45.0 ยท For Sequences 0.114.0
Installation guide
How your IT staff run Sequences on a server in your own network or in your AWS account: the container, the license file, a snapshot, sign-in with your own provider and upgrades.
What you need
Sequences comes as one container image that holds the search service, on Node.js 22, and the programs it runs: NCBI BLAST+ 2.17.0, FASTA36 36.3.8i, MAFFT 7.505, HMMER 3.4 with the antibody numbering models, and kscan 1.0.0, the scanner of Within k changes, built from its source with edlib 1.2.7. Queries, results and the sequence snapshot stay on your server, apart from two features that stay off until you turn them on. Watch emails go to the person's address through Resend. Each says how many new matches or publications one of their watches found and names no watch, query or match, unless you choose emails with details, which also send the watch's name and its new matches or publications. Claim suggestions send a case's claims, with the paragraphs of the description chosen for them, to a model you run. Otherwise, with sign-in on, the only connection the service opens is to your sign-in provider. See Network traffic.
- A Linux server (x86-64) with Docker Engine. For reference, 12 CPUs and 56 GiB of memory run the November 5, 2026 snapshot. A delta needs another 16 bytes of memory for each sequence it adds or changes and 16 bytes for each publication it lists, about 160 MB for 10 million sequences. Each server reads a delta's index into memory once, at the first search, export or download that needs it, and shares that copy with all that follow.
- Disk for the data volume: the snapshot (38 GiB for the November 5, 2026 build) and room for results, up to 8 GiB for one search unless you set less, and about 250 MB for a portfolio export at its limit. Loading a copy also needs room for the copy in an import folder, and a build like the November 5, 2026 one needs at least 110 GiB free in the volume. Docker keeps volumes under
/var/lib/docker/volumesunless you set it up otherwise, so check that disk. See Get a snapshot. - The Sequences image, a file you load with
docker load, and a license file, both from ConductScience. To get them, write to support@conductscience.com. - A snapshot: a copy with its SHA-256 list, or the public source files to build one.
- For sign-in: an OpenID Connect provider (Okta, Microsoft Entra ID, Keycloak or another), a host name with a TLS certificate, and a reverse proxy on the same server that serves HTTPS on that name.
- If you like, PubMed's files and the Retraction Watch Database for paper citations and marks, about 75 GB to download, PubTator 3's file of about 6 GB for a case's paper sentence, and ROR's data file for applicant suggestions. See Paper citations and marks and ROR suggestions.
- If you like, a language model on a server you control, for claim suggestions. See Claim suggestions from a model.
- If you like, the clinical context folder, a copy with its SHA-256 list from ConductScience, for the drugs, diseases and trials of a protein hit's target. See Clinical context.
- If you like, the funding trail folder and its SHA-256 list, from ConductScience. It holds the NIH awards and other grants of the papers listed under hits, and can hold the statements on federally sponsored research in US published applications. See Funding trail.
The commands below use <version> for the image's version, such as 0.43.1. They keep the service's files in two places: a Docker volume named sequences-data for data, and /etc/sequences on the server for settings, which the container reads at /config.
In your organization's own AWS account, the AWS install kit does these steps for you. See In your AWS account.
Load the image
Each release comes as two image files from the same code. sequences-<version>.tar is the standard image. sequences-<version>-fips.tar is the FIPS image, for a server whose policy requires FIPS 140 validated cryptography: it runs each algorithm the service uses in OpenSSL's FIPS provider, which holds a FIPS 140-3 certificate, apart from the license file's Ed25519 signature, which it checks outside the provider. Its tag is <version>-fips: write that tag where a command in this guide says <version>. The AWS kit runs the standard image only.
Check the file before you load it. Each release comes with SHA256SUMS, the SHA-256 of each of its files. In the folder that holds the files, this command prints OK for each file you have:
sha256sum -c --ignore-missing SHA256SUMS
From the first signed release on, a release also comes with SHA256SUMS.sig, ConductScience's signature of that list, and release-key.pub.pem, the release key. When yours has them, run these two first:
openssl pkey -pubin -in release-key.pub.pem -outform DER | openssl dgst -sha256 -r
openssl dgst -sha256 -verify release-key.pub.pem -signature SHA256SUMS.sig SHA256SUMS
The first prints the key's SHA-256: compare it with the one in ConductScience's guide Run Sequences on your own server, or ask support@conductscience.com. The second prints Verified OK. Load a file only after every check agrees. Then:
docker load -i sequences-<version>.tar
docker run --rm conductscience/sequences:<version> versions
versions prints the release and the version of each search program. In the FIPS image, fips-check runs each algorithm the service uses and says whether it ran in the FIPS provider. The service runs as user ID 10001 (sequences), not as root, and keeps everything it writes in /data. Only the backup and restore commands under Backups run the image as root.
The license file
The service needs a license file signed by ConductScience. The image checks the signature with ConductScience's public license key, which is built in, so there is no license server. The license names your organization, your site (a label for where it runs, such as "Main data center"), the plan (such as trial) and its first and last day. Both days count, in UTC. It names no computer, so it keeps working when you move the service to another server.
sudo install -d -m 0755 /etc/sequences
sudo install -m 0644 license.json /etc/sequences/license.json
docker run --rm -v /etc/sequences:/config:ro conductscience/sequences:<version> check-license
check-license prints what the file allows, and exits with 1 if it does not allow new searches today. From 60 days before the last day, it and the page say when the license ends. The page's footer names the plan and the last day, such as "Trial license, ends 2026-12-30", and so does the usage page.
An evaluation license, with the plan evaluation, lets your organization try the server edition before it buys. It runs the same service as any license: every feature, on as many servers and for as many people as you choose, because a license names no computer and counts no accounts. The footer and the usage page read "Evaluation license, ends" and its last day. When it ends, the server turns read-only, as after a trial: people can still open, mark and export their results, and new work needs a new license file. How long it runs, and what comes with it, are set in its terms.
At start, a license file that is missing or cannot be used stops the service, and the log says why. A license can give grace days: days after the last day on which new work still runs, while the page says when it stops.
Before the first day, and after the last day and any grace days, the service runs but refuses new work with a message that says why. That covers new searches of every kind, run again, alignments, antibody numbering, listing uploads, saving and editing sequences, creating or changing cases and their plans, making or resuming a watch, and portfolio exports. Watches stop running. People can still open, mark and export the results they have, download a case's report and a finished portfolio export, pause a watch, cancel a search and delete their own files. Put a renewed file in the same place: the service reads it again on the next request, with no restart. If the file goes missing or cannot be used while the service runs, new work pauses in the same way.
Get a snapshot
Searches run on a snapshot: a read-only folder built from public sequence files, with each sequence stored once with every record and publication that carries it. A snapshot's ID is its build date and a short hash, such as 2026-11-05-c456f6bdba29. A delta adds newer records to a snapshot without a rebuild. People see each delta as an addition: Collection on the search page offers Main plus New, and each watch runs again within the hour. In the data volume a snapshot lives in corpora/<snapshotId>/ and its deltas in deltas/<snapshotId>/<deltaId>/. Never change files there: the service checks them by their hashes.
There are two ways to get one: load a copy, or build one from the public files.
Load a copy
A copy comes as the snapshot folder and its SHA-256 list: one line for each file in the folder, with the SHA-256, two spaces and the path inside the folder. Put both in an import folder on the server:
/srv/sequences/import/2026-11-05-c456f6bdba29/
/srv/sequences/import/2026-11-05-c456f6bdba29.SHA256SUMS
The service reads the folder as user ID 10001, so make it readable by everyone:
sudo chmod -R a+rX /srv/sequences/import
You can check the files yourself first, with the standard tool:
cd /srv/sequences/import/2026-11-05-c456f6bdba29
sha256sum --check --quiet ../2026-11-05-c456f6bdba29.SHA256SUMS
Then load it into the data volume, which is made once and keeps everything the service stores:
docker volume create sequences-data
docker run --rm \
-v sequences-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/2026-11-05-c456f6bdba29 --sums /import/2026-11-05-c456f6bdba29.SHA256SUMS
load stops at the first problem:
- It checks
manifest.jsonagainst the list, and that the folder holds exactly the files the list names. - It copies each file into a hidden folder in the data volume, checking its SHA-256 as it copies.
- It runs the snapshot's own checks: its counts, its hashes and its BLAST databases.
- It makes the folder read-only and gives it its ID. Only then does the service list it.
If a step fails, the copy is removed and the message names the file or check that failed. A running service lists a new snapshot on the next page load, so loading needs no restart. Loading a snapshot that is already there is refused. A delta also comes as a folder and its SHA-256 list, and loads the same way, after its snapshot and every delta before it. The import folder and the data volume each need the size of the snapshot while it loads, and you can delete the import copy afterwards.
Check which release the service runs before you copy a delta to the server: a delta needs release 0.107.0 or later. /api/openapi.json gives the release, with no sign-in needed. With sign-in off, run this on the server:
curl -fsS http://127.0.0.1:8765/api/openapi.json | python3 -c 'import json, sys; print(json.load(sys.stdin)["info"]["version"])'
With sign-in on, the service answers only requests that name an allowed host, so run the same command with https://sequences.example.org/api/openapi.json, replacing sequences.example.org with your host name. Run load with an image of release 0.107.0 or later too. If the service runs an older release, upgrade first (see Upgrades), then copy and load the delta.
Sources and terms
Each source keeps its own terms, and the UniProt Consortium asks that its copyright statement goes with every copy of UniProt data. Each snapshot and delta carries ATTRIBUTION.txt: who supplies each source, the release, the terms page and the notices, including UniProt's statement word for word. The search page shows the same text under Sources and terms. If a snapshot folder has no ATTRIBUTION.txt, as in snapshots built before it existed, print the notice and keep it with your copy of the snapshot:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
attribution 2026-11-05-c456f6bdba29 > 2026-11-05-c456f6bdba29.ATTRIBUTION.txt
Build from the public files
fetch downloads the public files and build turns them into a snapshot. fetch, crossref and antibody-names fetch (Antibody names) are the only commands that download anything. fetch reads from https://ftp.ncbi.nlm.nih.gov/genbank/, https://ftp.ebi.ac.uk/pub/databases/ and https://ftp.uniprot.org/pub/databases/uniprot/. GenBank publishes only its current release, so fetch needs that release number:
curl -s https://ftp.ncbi.nlm.nih.gov/genbank/GB_Release_Number
With the number it prints, such as 274.0:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
fetch --source all --release 274.0 --out /data/inputs/release-274.0
docker run --rm -e UV_THREADPOOL_SIZE=10 -v sequences-data:/data conductscience/sequences:<version> \
build --inputs /data/inputs/release-274.0 --title "Public patent sequences" --workers 8
--source allfetches the patent files and Swiss-Prot. TrEMBL and the ESTs are large, soallleaves them out. To fetch them too, add--source uniprot-trembl --source ena-estto the samefetchcommand.fetchopens at most four connections to one server, records each file's size and SHA-256, and picks up where it stopped if you run it again after an interruption.buildchecks every file against that record, then parses, merges and indexes them, builds the BLAST databases, runs its checks and seals the snapshot. It prints the snapshot ID at the end.- Set
UV_THREADPOOL_SIZEto the number of workers plus two.--resumerestarts a failed build after its last finished stage.
As a guide, from the November 5, 2026 build of the patent sets and Swiss-Prot: the source files take about 25 GB, the build needs up to 79 GiB more working space in the data volume while it runs, and the finished snapshot keeps 38 GiB. Before you run fetch, leave at least 110 GiB free in the volume, which covers the source files and the build together. When the snapshot is built, and its source files are scanned for the species of each hit (see Species of each hit), the source files can go:
docker run --rm -v sequences-data:/data --entrypoint rm conductscience/sequences:<version> \
-rf /data/inputs/release-274.0
Check a snapshot again
docker run --rm -v sequences-data:/data conductscience/sequences:<version> verify 2026-11-05-c456f6bdba29
docker run --rm -v sequences-data:/data conductscience/sequences:<version> deltas 2026-11-05-c456f6bdba29
verify runs the same checks as the load, and exits with 1 if any fails. deltas lists a snapshot's deltas in order. sums <snapshotId> prints the folder's SHA-256 list, to compare with the list that came with the copy.
Start the service
Sign-in is on when BIOSEQUENCE_ALLOWED_HOSTS is set, and off when it is not. Choose one of the two ways to run the service. Both use the sequences-data volume with a snapshot in it and the license file in /etc/sequences.
| What | Sign-in off | Sign-in on |
|---|---|---|
| Who uses it | One person, or a test, on one Linux machine: the local workbench | People in your organization, through your sign-in provider |
| Where it answers | 127.0.0.1 of that machine only | Your host name, through your HTTPS reverse proxy |
| Jobs | One shared list. Anyone who can reach port 8765 on that machine, such as another local user or an SSH tunnel, can open every search | Each person's searches belong to their account, and nobody else can open them |
| Collections | Snapshot sets, and FASTA collections imported on the page | Snapshot sets, with no imports |
Sign-in off: one Linux machine
docker run -d --name sequences --restart unless-stopped \
--network host \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges \
-v sequences-data:/data \
-v /etc/sequences:/config:ro \
conductscience/sequences:<version>
Open http://127.0.0.1:8765/search on that machine (http://localhost:8765/search works too). From another computer, tunnel the port over SSH and open the same address there:
ssh -L 8765:127.0.0.1:8765 <user>@<server>
With sign-in off the service listens on 127.0.0.1 only and answers only requests addressed to 127.0.0.1 or localhost. --network host puts that address on the server itself, since a published port (-p) cannot reach a service that listens on the container's own loopback. This is for Linux servers. Docker Desktop on Windows and macOS handles host networking differently, and this guide does not cover it.
Sign-in on: your organization
You need a host name, such as sequences.example.org, its TLS certificate, and a reverse proxy on the same server that serves HTTPS on that name.
1. Register Sequences with your provider
Register a web application (a confidential client) that uses the authorization code flow:
- Redirect URI:
https://sequences.example.org/api/auth/callback - Scopes:
openid email profile - Client authentication: a client secret
Note the issuer URL, the client ID and the client secret. Provider settings shows where each one is.
2. Write the settings
Save the client secret in its own file, readable only by root and by group 10001, the service's group. read asks for the secret without showing it, so it stays out of your shell history:
sudo install -m 0640 -g 10001 /dev/null /etc/sequences/oidc-client-secret
read -rs SECRET && printf %s "$SECRET" | sudo tee /etc/sequences/oidc-client-secret > /dev/null; unset SECRET
Then write the settings in /etc/sequences/sequences.env:
BIOSEQUENCE_ALLOWED_HOSTS=sequences.example.org
BIOSEQUENCE_OIDC_ISSUER=<issuer URL>
BIOSEQUENCE_OIDC_CLIENT_ID=<client ID>
BIOSEQUENCE_OIDC_CLIENT_SECRET_FILE=/config/oidc-client-secret
BIOSEQUENCE_OIDC_NAME=your Example Org account
BIOSEQUENCE_ALLOWED_DOMAINS=example.org
Choose who may sign in with at least one of these:
BIOSEQUENCE_INVITE_FILE=/config/invite.txt: a file with one person per line, as the user claim names them (their email address by default), where#starts a comment. The service reads the file again whenever it changes and checks it on every request, so removing a line ends that person's access at their next click.BIOSEQUENCE_ALLOWED_DOMAINS: everyone whose user claim ends in@and one of these domains.BIOSEQUENCE_OIDC_REQUIRED_GROUP: only people whose group claim holds this value. The group claim is read at sign-in, so a change at the provider applies at the next sign-in, withinBIOSEQUENCE_SESSION_HOURS(12 by default).
A person may sign in when the invite file or an allowed domain names them, if you set either, and their group claim holds the required group, if you set one. With only a required group, everyone in that group may sign in. To let someone see the usage page, add them to BIOSEQUENCE_ADMINS as the user claim names them, separated by commas, such as BIOSEQUENCE_ADMINS=lab.manager@example.org. They must still be allowed to sign in. See Usage.
- With
BIOSEQUENCE_OIDC_USER_CLAIMset to anything butemail, also setBIOSEQUENCE_OIDC_REQUIRED_GROUP. The service checks that an email address is verified, but it has no way to check another claim, such as a sign-in name a user can change at some providers. - A user value with letters beyond plain ASCII, such as an accented name, never matches the invite file or an allowed domain. To let such people in, use the required group alone, with no invite file and no allowed domains.
- A case's owner can hand the case to a colleague's account. See Hand a case to a colleague. The colleague must have signed in here once, and the rule above must let them in now. The required group is the one their last sign-in found, as for API tokens, and a refused sign-in takes it away. With
BIOSEQUENCE_OIDC_USER_CLAIM=email, a case goes only to an address at its owner's own email domain, even whenBIOSEQUENCE_ALLOWED_DOMAINSnames several. - With more than one name in
BIOSEQUENCE_ALLOWED_HOSTS, sign-in happens at one address only:BIOSEQUENCE_SITE_URL, or the first name in the list. A visit on another name goes to that address first. Register only that address's redirect URI. - If your provider's certificate comes from your own certificate authority, put the authority's PEM file in
/etc/sequences/ca.pemand addNODE_EXTRA_CA_CERTS=/config/ca.pemto the settings.
3. Start the container
If you started the sign-in-off container, remove it first: docker stop sequences && docker rm sequences. Then:
docker run -d --name sequences --restart unless-stopped \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges \
-p 127.0.0.1:8765:8765 \
-v sequences-data:/data \
-v /etc/sequences:/config:ro \
--env-file /etc/sequences/sequences.env \
conductscience/sequences:<version>
docker logs sequences
The log names the host, the issuer and the license, for example:
Sequences for sequences.example.org, signed in through https://login.example.org/realms/example, on 0.0.0.0:8765
Trial license for Example Org, 2026-12-01 to 2027-01-15
Inside the container the service listens on all of its addresses (BIOSEQUENCE_HOSTED_BIND=0.0.0.0), and -p 127.0.0.1:8765:8765 publishes that port on the server's loopback address only. Keep the 127.0.0.1: part, so that only the reverse proxy reaches the port. The service answers only requests that name an allowed host. A request addressed to 127.0.0.1 works only from inside the container, which is how the image's health check reaches it. The one exception is GET /api/health, which also answers at an IP address, for a health check that cannot name your host. With this command, such a check runs on the server itself, at http://127.0.0.1:8765/api/health. A load balancer on another machine needs the port published on an address it can reach, as the AWS kit publishes it on the server's private address. See Health and logs.
4. Point the reverse proxy at it
The proxy serves HTTPS on your host name, passes the Host header through unchanged, and forwards to http://127.0.0.1:8765. With Caddy, which keeps the Host header and can get its own certificate for a name that is reachable from the internet:
sequences.example.org {
reverse_proxy 127.0.0.1:8765
}
For an internal name, give Caddy your certificate with a tls line in the same block, such as tls /etc/ssl/sequences.example.org.crt /etc/ssl/sequences.example.org.key.
With nginx, where a large sequence listing or result file needs a large upload limit:
server {
listen 443 ssl;
server_name sequences.example.org;
ssl_certificate /etc/ssl/sequences.example.org.crt;
ssl_certificate_key /etc/ssl/sequences.example.org.key;
client_max_body_size 2g;
proxy_request_buffering off;
location / {
proxy_pass http://127.0.0.1:8765;
proxy_set_header Host $host;
}
}
Open https://sequences.example.org/ and sign in.
What sign-in checks
The service uses the authorization code flow with PKCE (S256), a one-time state and a nonce. It reads the provider's discovery document and keys, checks the ID token's signature (RS256, PS256, ES256 and EdDSA families, never none or a shared secret), its issuer, audience, expiry and nonce, and asks the userinfo endpoint for claims the ID token leaves out. With BIOSEQUENCE_OIDC_USER_CLAIM=email, the provider must mark the email as verified. Sessions are signed cookies (__Host-, Secure, HttpOnly, SameSite=Lax) that last BIOSEQUENCE_SESSION_HOURS. The service keeps a record of each session in sessions/ on the data volume: its account, when its sign-in ends and the working files it is making (see Working files). Each search belongs to the account's subject ID at the provider, and any other account gets "not found" for it.
Five minutes before a session ends, the page warns the person and offers to sign in again in a new tab, so the work open on the page is kept. Signing in again is a new sign-in at your provider: the rule for who may sign in is checked again, and the new session lasts BIOSEQUENCE_SESSION_HOURS from then. No session lasts longer than that setting. Signing in again before the end keeps the session's working files.
Signing out removes the session from that browser and deletes its working files. It does not sign the person out of the provider. Removing someone from the invite file ends their access at their next click. To end every session at once, for example when someone leaves and the rule that let them in is a domain or a group, replace the cookie key and restart, and everyone signs in again:
docker exec sequences rm /data/session-secret
docker restart sequences
If you set BIOSEQUENCE_SESSION_SECRET_FILE, put a new key of 32 characters or more in that file instead, then restart. A new cookie key does not end API tokens: revoke them on the usage page.
Move to another provider
Each search belongs to the user ID (sub) that your provider gave its owner, and another provider's IDs name other people. So the data volume keeps the provider it first signed in through: the first start with sign-in on writes the issuer to /data/sign-in-provider.json, and a later start with a different BIOSEQUENCE_OIDC_ISSUER stops with a message that names both. To move on purpose, choose one:
- Start fresh. Use a new data volume with the new provider and load the snapshot into it. Keep the old volume if people need their old searches, and open it with the old provider.
- Keep the volume. Stop and remove the container, remove the record with the commands below, put the new provider's settings in
/etc/sequences/sequences.envand start the container as in step 3. Old searches stay on the volume under the old user IDs. If the new provider gives people the same user IDs (sub), for example because it reads the same user directory, they keep their searches. Otherwise nobody can open the old ones.
docker stop sequences && docker rm sequences
docker run --rm -v sequences-data:/data --entrypoint rm conductscience/sequences:<version> /data/sign-in-provider.json
API tokens for programs
A program, such as a script your search team runs, signs in with a personal API token instead of a browser. A signed-in user chooses API tokens from their account button, names the token, picks how long it lasts and copies it. The page shows it once. The program sends it on every request:
curl -H "Authorization: Bearer <token>" https://sequences.example.org/api/jobs
https://sequences.example.org/api/openapi.json describes the routes a program can use, in OpenAPI 3.1. It is open without sign-in and holds no data. POST /api/batches starts up to 100 searches in one request.
- A token acts as the person who made it, under the same rules, checked on every request. Taking someone off the invite file, or their domain off
BIOSEQUENCE_ALLOWED_DOMAINS, stops their tokens at the program's next request. - The group claim is read only at a browser sign-in. With
BIOSEQUENCE_OIDC_REQUIRED_GROUPset, a token keeps the groups its owner had when it was made, so when someone leaves the group, revoke their tokens on the usage page. - A token lasts at most
BIOSEQUENCE_TOKEN_DAYSdays, 90 by default, and each person may have 10 at a time. The service keeps only a SHA-256 hash of each token, in/data/tokens.json, so nobody can read one back. - A token cannot make another token: that needs the browser.
BIOSEQUENCE_TOKEN_DAYS=0turns tokens off and stops every token at once. Turned on again, tokens that have not expired work again, so to end tokens for good, revoke them.- A request with a token skips the browser's cross-site checks, which protect the cookie a browser sends by itself. It still has to name an allowed host.
With sign-in off there are no tokens: a program on the same machine calls http://127.0.0.1:8765 as the page does.
Connectors for document systems
A connector lets your document system send an application's claims and sequence listing into a case. The user guide's Connectors section is for the person who sets one up: the connector token, the manifest, the answer and the folder program. For the server:
- The people in
BIOSEQUENCE_ADMINSmake connector tokens on the API tokens page, with sign-in on.BIOSEQUENCE_TOKEN_DAYSsets how long they may last, as for API tokens, and0turns them off too. POST /api/connector/importstakes a whole document set in one request: a listing up toBIOSEQUENCE_LISTING_BYTESand up to four files of claims of 15 MiB each. The nginx example above, withclient_max_body_size 2g, takes that. Behind another proxy, let this route take at least 320 MB.- While a call arrives, its files wait in
/data/connector/incoming/, which empties when the call answers and at each start./data/connector/sets/records which case each source and ID went into. - The folder program is in the image at
/app/scripts/sequences-connector.mjs, and the server hands it out at/docs/sequences-connector.mjs.
Provider settings
Okta
- In the Okta admin console, go to Applications, Create App Integration, and choose OIDC, OpenID Connect and Web Application.
- Grant type: Authorization Code. Sign-in redirect URI:
https://sequences.example.org/api/auth/callback. - Copy the Client ID and Client secret from the application's General tab.
- To limit access to a group, open Security, API, the
defaultauthorization server and Claims, and add a claim namedgroupsto the ID token, of value type Groups, with a filter that matches your group.
BIOSEQUENCE_OIDC_ISSUER=https://example.okta.com/oauth2/default
BIOSEQUENCE_OIDC_CLIENT_ID=<client ID>
BIOSEQUENCE_OIDC_CLIENT_SECRET_FILE=/config/oidc-client-secret
BIOSEQUENCE_OIDC_NAME=your Example Org Okta account
BIOSEQUENCE_OIDC_REQUIRED_GROUP=Sequences Users
The issuer of the default authorization server ends in /oauth2/default. With Okta's org authorization server it is the Okta domain alone, such as https://example.okta.com.
Microsoft Entra ID
- In the Microsoft Entra admin center, go to App registrations and New registration. Add a Web redirect URI:
https://sequences.example.org/api/auth/callback. - Under Certificates & secrets, add a client secret and copy its value.
- To limit access, add an app role under App roles, such as
Sequences.User, then assign people or groups to it under Enterprise applications, the application, Users and groups. Turning on Assignment required there also stops everyone else at the Microsoft sign-in.
Entra ID tokens carry no email_verified claim, so name people by their sign-in name instead:
BIOSEQUENCE_OIDC_ISSUER=https://login.microsoftonline.com/<tenant ID>/v2.0
BIOSEQUENCE_OIDC_CLIENT_ID=<application (client) ID>
BIOSEQUENCE_OIDC_CLIENT_SECRET_FILE=/config/oidc-client-secret
BIOSEQUENCE_OIDC_CLIENT_AUTH=client_secret_post
BIOSEQUENCE_OIDC_NAME=your Example Org Microsoft account
BIOSEQUENCE_OIDC_USER_CLAIM=preferred_username
BIOSEQUENCE_OIDC_GROUP_CLAIM=roles
BIOSEQUENCE_OIDC_REQUIRED_GROUP=Sequences.User
Keycloak
- In your realm, go to Clients and Create client: type OpenID Connect, client ID
sequences. - Turn Client authentication on and keep Standard flow. Valid redirect URI:
https://sequences.example.org/api/auth/callback. - Copy the secret from the client's Credentials tab.
- To limit access to a group, open the client's Client scopes tab, then
sequences-dedicated, Add mapper, By configuration and Group Membership. Set the token claim name togroups, with Add to ID token on. With Full group path on, the value is the group's path, such as/sequences-users.
BIOSEQUENCE_OIDC_ISSUER=https://keycloak.example.org/realms/<realm>
BIOSEQUENCE_OIDC_CLIENT_ID=sequences
BIOSEQUENCE_OIDC_CLIENT_SECRET_FILE=/config/oidc-client-secret
BIOSEQUENCE_OIDC_NAME=your Example Org account
BIOSEQUENCE_OIDC_REQUIRED_GROUP=/sequences-users
Keycloak sends email_verified from each user's profile, so people need a verified email address, or set BIOSEQUENCE_OIDC_USER_CLAIM=preferred_username.
Other providers
Any provider that publishes an OpenID Connect discovery document at <issuer>/.well-known/openid-configuration and supports the authorization code flow works with the same settings. If the provider sends the user claim or the groups only from its userinfo endpoint, the service asks for them there.
In your AWS account
The AWS install kit does the steps of this guide for you in your organization's own AWS account. It runs Sequences 0.43.0 and later. On November 14, 2026, ConductScience installed the kit in an AWS account, and every check passed. On November 15, 2026, it tested kit 1.2.0's workers in an AWS account, with at most 2: AWS added them while searches waited, each batch of 20 searches took 340 to 381 seconds with both, where the server alone took 925, and AWS removed them 9 minutes after the last search ended.
The kit is three files from ConductScience: a CloudFormation template, kit.sh, the script the server runs, and the kit's own guide, for your AWS administrator. The kit's guide gives every step: what goes where in your bucket, the secrets, the network the server needs, each parameter, updates, backups and removal. One stack made from the template creates:
- One EC2 server, memory-optimized, with no public address and no key pair. The default,
r6i.2xlarge, has 8 vCPUs and 64 GiB of memory, and the service gets all of it but 8 GiB. - Encrypted gp3 disks: a 30 GiB root disk, and a data volume of 200 GiB by default for the snapshots and everything the service stores.
- Security groups, and a role for the server. It can read only its folder of your S3 bucket, the stack's settings and the secrets you name, and with Amazon ECR pull from one repository. Policies you name add to it.
- With sign-in, an HTTPS Application Load Balancer with your ACM certificate. It is internal by default, so only your network reaches it.
- With workers, from kit 1.2.0: a group of worker servers (see Several servers) that AWS adds while searches wait, up to the number you set, and removes once no search has run for 5 minutes. Workers need sign-in through the load balancer and Sequences 0.92.0 or later.
On its first boot the server installs Docker, loads the Sequences image from your bucket or from Amazon ECR, copies each snapshot from the bucket, checks every file against the snapshot's SHA-256 list and runs verify on it, then starts the service. The stack is complete only when the service runs. It waits up to 12 hours for the first boot, which takes longer for a larger snapshot. If a step fails, the service does not start and the stack fails, and the server's console says why. You provide the network (the VPC, its subnets, and a NAT gateway or VPC endpoints), the bucket, the secrets, the certificate, the DNS record and the registration at your sign-in provider.
People reach Sequences in one of two ways:
- Through the load balancer, the default: people open your host name and sign in with your OpenID Connect provider.
- Through port forwarding, for a test or one person: sign-in is off, and you open the page through a Session Manager port forward from your computer. Anyone allowed to start a Session Manager session on the server can open Sequences. With sign-in off there are no accounts, so they all see the same saved work.
You manage the server through AWS Systems Manager Session Manager. Nothing can reach its port 22. To change the install, such as to add a delta or move to a new release, put any new files in the bucket, update the stack, then run kit.sh apply on the server in a Session Manager shell, which puts the update in place. A later kit goes in the same way, with its new template and kit.sh. kit.sh check runs in one go the checks this guide gives one at a time: versions, check-license, the health checks and verify. The service starts again 10 seconds after any stop, as when its health check stops it (see Health and logs). Back up the data volume with EBS snapshots, through AWS Backup or Amazon Data Lifecycle Manager.
The kit sets only the settings its parameters name, so an install made with it has no paper citations and marks, ROR suggestions, watch emails, claim suggestions or add-ons. ConductScience has no access to your account, and the server sends nothing to ConductScience. To get the kit, write to support@conductscience.com.
Several servers
One Sequences install can run on several servers that share one queue. One server, the coordinator, holds the pages, sign-in, the queue and every result, as a single server does. The others are workers: each takes the coordinator's searches of a snapshot, runs them and sends the results back, and the coordinator checks every result before the search completes. People use the coordinator's address as before. A worker has no page and no public port, reads no license file and keeps no audit log. BIOSEQUENCE_ROLE sets each server's part: single, the default, coordinator or worker. Several servers need 0.81.0 or later, the same release on every server.
Workers help when searches of a snapshot wait for each other: many people searching at once, case runs with many searches, or batches from programs. Each worker runs as many searches at once as its BIOSEQUENCE_CONCURRENCY says, on its own CPUs and memory. On one server with CPUs to spare, BIOSEQUENCE_CONCURRENCY=2 or more runs searches side by side with nothing more to set up. Only searches of a snapshot go to workers. Searches of an attached database, Align exactly, antibody numbering, multiple alignments, portfolio exports and result files imported into a case run on the coordinator.
A coordinator keeps its searches of a snapshot through a restart: a waiting search keeps its place, and a search the coordinator ran itself starts again. A search a worker runs goes on if the worker reaches the coordinator within a minute of the coordinator's start, and otherwise waits in the queue again. The coordinator's other work ends at a stop, as on a single server. The time a combined search or a case run has left counts the searches the workers run.
Measured with the November 13, 2026 snapshot on release 0.92.0, in the load test of Searches an hour, with every server on one 32-CPU host and each on CPUs of its own: each worker 6, which are 3 cores, and the coordinator 2, running no search. With 100 searches at once, two workers ran 552 searches an hour and three ran 828. With 20 at once, two and three workers took 120 and 97 seconds, where a single server with 8 CPUs, running one search at a time on 6 threads, took 198. Each search took longer on a worker's 3 cores, but two or three ran at once. When a worker was killed, stopped or frozen in the middle of a search, or the coordinator's service was killed or frozen, all 20 searches still completed, each once, and a search cut off on a worker started again on another. Their hits matched the other runs of each query. Smith-Waterman searches estimate their E-values from a random sample, so the rows nearest their E-value cut can differ a little between any two runs, with or without a failure.
What each server needs
- The coordinator runs with sign-in on, the license file and the snapshots and deltas, as a single server does. See Sign-in on: your organization. At
BIOSEQUENCE_CONCURRENCY=0it runs no search of a snapshot itself, so it needs fewer CPUs than a worker. It keeps one slot for the work that stays on it. - Each worker runs the coordinator's image and holds the same snapshots and deltas in its own data volume, loaded from the same copies, with the same paper folder and taxa folder if the coordinator has them. It needs no license file, no sign-in settings and no published port.
- Every server has the worker key: one random value of 32 characters or more, the same on all of them. Anyone with the key can take searches and read their queries, so keep it as safe as the client secret. To change it, start every server again with the new key.
- Give every worker the same CPUs, memory and
BIOSEQUENCE_THREADS. The coordinator's time estimates follow the last search on any server, and workers of one size keep them steady.
Load each new snapshot or delta on every worker first, then on the coordinator. People search what the coordinator lists, and at BIOSEQUENCE_CONCURRENCY=0 a search waits until a worker holds its copy. The coordinator and its workers look at their copies every minute, so a new copy takes searches within a minute, with no restart.
Before a worker takes a search, the coordinator checks that it runs the same release, the same search programs and the same paper and taxa folders. It also compares the SHA-256 list of each snapshot and delta the worker holds with its own, and gives the worker only searches of the snapshots and deltas that match. load keeps the list it checked each file against, so a copy it loaded is ready at once. A snapshot built on the server, or copied in any other way, is read whole at the first start, which can take minutes for a large one. verify stays the full check of a copy. A worker uses the coordinator's time limit for one search and its largest result file (BIOSEQUENCE_JOB_SECONDS, BIOSEQUENCE_MAX_OUTPUT_BYTES), with its own threads.
Settings for several servers
| Setting | Where | Default | What it does |
|---|---|---|---|
BIOSEQUENCE_ROLE | Every server | single | single, coordinator or worker. Any other value stops the start. A coordinator needs sign-in on |
BIOSEQUENCE_WORKER_KEY_FILE | Every server | none | A file that holds the worker key, such as /config/worker.key. BIOSEQUENCE_WORKER_KEY takes the key itself instead. The key is never printed or logged |
BIOSEQUENCE_CONCURRENCY | Every server | 1 | On a worker, the searches it runs at once, 1 to 16. On the coordinator, 0 to 16: at 0 its workers run every search of a snapshot, and above 0 it runs that many itself beside them |
BIOSEQUENCE_WORKER_BIND | Coordinator | 127.0.0.1 | The address inside the container where the worker port listens. 0.0.0.0 for workers on the coordinator's Docker network or on other hosts |
BIOSEQUENCE_WORKER_PORT | Coordinator | 8766 | The worker port, 1024 to 65535, other than BIOSEQUENCE_PORT |
BIOSEQUENCE_WORKER_TLS_CERT, BIOSEQUENCE_WORKER_TLS_KEY | Coordinator | none | A PEM certificate and its private key, which turn on TLS on the worker port. Set both or neither |
BIOSEQUENCE_COORDINATOR_URL | Worker | none | The coordinator's worker address with no path, such as https://coordinator.example.org:8766. http:// works only for the same machine or a one-word host on the same Docker network, such as http://sequences:8766 |
BIOSEQUENCE_WORKER_NAME | Worker | the container's ID | The name the coordinator's log and the usage page show: 1 to 63 letters, digits, dots, dashes or underscores, starting with a letter or digit. Each worker needs its own |
A setting that is missing or does not fit stops the start, before the data volume opens, with a message that names it. The coordinator's audit log notes the settings of each start, with the keys shown as set.
Two workers on one host
Two workers beside the coordinator on its own server are the simplest setup, and a way to try several servers before you add hosts. The coordinator and its workers join a Docker network of their own, sequences-workers, where each worker reaches the worker port by the coordinator's container name. The port is not published, so only containers on that network reach it, and it needs no TLS there. The network outlives each container, so after the coordinator restarts, its workers join it again by themselves. The commands below give the coordinator 2 CPUs and each worker 6.
First make the worker key, readable only by root and by group 10001, the service's group, and the network:
sudo install -m 0640 -g 10001 /dev/null /etc/sequences/worker.key
head -c 32 /dev/urandom | base64 | sudo tee /etc/sequences/worker.key > /dev/null
docker network create sequences-workers
Load each snapshot and delta the coordinator holds into a volume for each worker, from the same copies, as in Load a copy. For the first worker:
docker volume create sequences-w1-data
docker run --rm \
-v sequences-w1-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/2026-11-05-c456f6bdba29 --sums /import/2026-11-05-c456f6bdba29.SHA256SUMS
If the coordinator has a paper folder or a taxa folder, copy it into each worker's volume as it is, since a folder made again is another folder, and give the worker the same BIOSEQUENCE_PAPERS or BIOSEQUENCE_TAXA with one more -e each in its command below. For a taxa folder, run this again with /from/taxa in place of /from/papers:
docker run --rm --user 0 \
-v sequences-data:/from:ro \
-v sequences-w1-data:/to \
--entrypoint cp conductscience/sequences:<version> -a /from/papers /to/
Add these lines to the coordinator's /etc/sequences/sequences.env:
BIOSEQUENCE_ROLE=coordinator
BIOSEQUENCE_WORKER_KEY_FILE=/config/worker.key
BIOSEQUENCE_CONCURRENCY=0
BIOSEQUENCE_WORKER_BIND=0.0.0.0
Start the coordinator as in Start the container, with --cpus 2 --network sequences-workers added. If it runs, remove it first with docker stop sequences && docker rm sequences. Then start the two workers:
for w in w1 w2; do
docker run -d --name sequences-$w --restart unless-stopped \
--network sequences-workers --cpus 6 \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges \
-v sequences-$w-data:/data \
-v /etc/sequences/worker.key:/config/worker.key:ro \
-e BIOSEQUENCE_ROLE=worker \
-e BIOSEQUENCE_WORKER_NAME=$w \
-e BIOSEQUENCE_COORDINATOR_URL=http://sequences:8766 \
-e BIOSEQUENCE_WORKER_KEY_FILE=/config/worker.key \
-e BIOSEQUENCE_THREADS=6 \
conductscience/sequences:<version>
done
A worker mounts the key alone, not the rest of /etc/sequences. Its log names the coordinator and says when its copies match:
Sequences worker w1 for the coordinator at http://sequences:8766: 1 search at a time, 6 threads each. Data stays on this computer: /data
Joined the coordinator at http://sequences:8766 as w1.
This worker's copy of snapshot 2026-11-05-c456f6bdba29 matches the coordinator's.
The coordinator's log says how many searches came back from its last stop, and when each worker joins:
Coordinator: searches of a snapshot keep their place in the queue across a restart (0 came back from the last stop).
Coordinator: BIOSEQUENCE_CONCURRENCY is 0, so workers run every search of a snapshot and this server runs none.
Workers join on 0.0.0.0:8766.
The worker port takes plain HTTP. For workers on other hosts, set BIOSEQUENCE_WORKER_TLS_CERT and BIOSEQUENCE_WORKER_TLS_KEY.
Worker w1 joined: 1 search at a time, 6 threads each.
Worker w1 matches this server's copy of snapshot 2026-11-05-c456f6bdba29.
Worker w2 joined: 1 search at a time, 6 threads each.
Worker w2 matches this server's copy of snapshot 2026-11-05-c456f6bdba29.
The plain HTTP line is expected here: the worker port is not published, so only the sequences-workers network reaches it.
The workers keep the network while the coordinator stops and starts, so a restart of the coordinator needs nothing done to them, whether you restart it, Docker does after a crash or a reboot, or its health check does (see Health and logs). Their logs say that it does not answer, they say hello every 20 seconds, and they join it again once it answers. A search a worker runs goes on if the worker reaches the coordinator within a minute of its start, and otherwise waits in the queue again, counted as cut off once.
When you make the coordinator's container anew, as in an upgrade, start it with --network sequences-workers again. The workers must run its release, so drain them before the upgrade (see Drain and stop a worker), remove them with docker stop sequences-w1 sequences-w2 && docker rm sequences-w1 sequences-w2, and run the loop again with the new version once the coordinator runs.
Workers on other hosts
Workers on other hosts reach the coordinator's worker port across your network, so turn on TLS there: each request carries the worker key, and each search's queries and results cross the network. You need a certificate for the name the workers use, such as coordinator.example.org, and its private key, as PEM files.
On the coordinator, put both files in /etc/sequences, the private key readable only by root and group 10001:
sudo install -m 0644 worker-tls.crt /etc/sequences/worker-tls.crt
sudo install -m 0640 -g 10001 worker-tls.key /etc/sequences/worker-tls.key
Then add these lines to its settings:
BIOSEQUENCE_WORKER_BIND=0.0.0.0
BIOSEQUENCE_WORKER_TLS_CERT=/config/worker-tls.crt
BIOSEQUENCE_WORKER_TLS_KEY=/config/worker-tls.key
Publish the worker port on the address the workers reach, with -p <coordinator address>:8766:8766 in its docker run, and let your firewall admit only the workers' addresses to it. The log then says Workers join on 0.0.0.0:8766 over TLS. Without TLS on that address, it says that the worker port takes plain HTTP.
On each worker host, copy the coordinator's key to /etc/sequences/worker.key, readable only by root and group 10001, and put the PEM file of your certificate authority in /etc/sequences/ca.pem if your own authority signed the certificate. Load the snapshots and deltas into the host's sequences-data volume from the same copies, and copy the coordinator's papers/ and taxa/ folders into it as they are if the coordinator has them (see Backups), with the same BIOSEQUENCE_PAPERS and BIOSEQUENCE_TAXA in the worker's command. Then start the worker. Leave out NODE_EXTRA_CA_CERTS when a public authority signed the certificate:
docker run -d --name sequences --restart unless-stopped \
--read-only --tmpfs /tmp \
--cap-drop ALL --security-opt no-new-privileges \
-v sequences-data:/data \
-v /etc/sequences:/config:ro \
-e BIOSEQUENCE_ROLE=worker \
-e BIOSEQUENCE_WORKER_NAME=w3 \
-e BIOSEQUENCE_COORDINATOR_URL=https://coordinator.example.org:8766 \
-e BIOSEQUENCE_WORKER_KEY_FILE=/config/worker.key \
-e NODE_EXTRA_CA_CERTS=/config/ca.pem \
conductscience/sequences:<version>
A worker opens no port. It connects to its coordinator and to nothing else. It takes an http:// address only for its own machine or a one-word host on the same Docker network, such as a container named coordinator, so the key never crosses a network in plain text by mistake.
Drain and stop a worker
To take a worker out without cutting off a search, drain it, then stop it:
docker exec sequences-w1 /opt/node/bin/node /app/services/biosequence/container/start.mjs drain
docker stop sequences-w1
drain asks the worker to finish the searches it runs and take no more. It names the searches it waits for, and it ends with 0 once the worker runs none:
Asked worker w1 to drain: it finishes the searches it runs and takes no more.
Waiting for the search it runs: <search ID>.
Worker w1 is drained: it runs no search. Stop it now.
It ends with 1 at once on a server that is not a worker, or when no worker runs on that volume, and later if the worker stops before it drains. A search that reaches the worker after the drain began goes back to the queue at once. The coordinator's log and the usage page show the worker as draining, then drained. A drained worker takes no search until it starts again.
It also ends with 1 if the worker starts again before it drains, since a start clears the drain and the worker takes searches again, and after 60 minutes, or the minutes drain --wait <minutes> gives, from 1 to 1440. Then it says what the worker last said, and the drain stays asked, so the worker takes no more searches:
Stopped waiting after 60 min. Worker w1's status file, written 4.2 s before, said: Draining, with the search it runs: <search ID>.
The drain stays asked, so the worker takes no more searches. Run drain again to wait longer, or stop the worker, which gives back the searches it runs.
A docker stop without a drain gives back the searches the worker runs: they wait in the queue again, in their place, and start over on the next free server. A worker that stops answering, as after a crash or a network fault, is lost after 60 seconds, and its searches wait in the queue again too. A search cut off three times this way fails with "The search was cut off 3 times before it finished. Submit it again." A stop or a drain never counts as a cut.
The Servers section
On a coordinator, the usage page ends with a Servers section for its admins. A line counts the workers that take searches and how many searches they run at once. Then a row for each worker gives its name; its state: Starting, Checking its snapshot, Ready, Busy, Waiting for the coordinator, Draining, Drained, Lost, or Refused with the reason, such as "Refused: another release"; the searches it runs; how many it runs at a time; the snapshot and delta copies that match the coordinator's, with any that differ under them; and when the coordinator last heard from it. Reload the page to read it again.
GET /api/admin/workers gives the same list to an admin's program, and GET /api/status on a coordinator adds servers: the workers that take searches now and how many searches of a snapshot run at once.
When the coordinator refuses a worker
The worker's log says what to change, and a refused worker fails its health check. The coordinator logs each refusal once, notes it in its audit log and shows the worker on the usage page as Refused, with the reason. A worker with another key is never named there: the coordinator's audit log notes its requests without a name. Fix the cause and start the worker again. A worker refused for its name joins by itself once the other worker of that name has been silent for 30 seconds, as after a crash.
| Refusal | What to change |
|---|---|
| Another release, other search programs, or other search settings | Run one image on every server |
| Another paper folder | Copy the coordinator's paper folder into the worker's volume as it is, and set the same BIOSEQUENCE_PAPERS |
| Another taxa folder | Copy the coordinator's taxa folder into the worker's volume as it is, and set the same BIOSEQUENCE_TAXA |
| Its name is in use | Give each worker its own BIOSEQUENCE_WORKER_NAME |
| The worker key | Use the same worker key on every server |
A snapshot or delta copy that differs from the coordinator's is not a refusal: the worker takes no search of it and goes on with the searches of its other copies. Its log names up to five files that differ, and the usage page lists the copy under Differs. Load that copy again from the files you received.
Settings
Set these with -e NAME=value or in an env file (--env-file). Docker reads an env file literally: write NAME=value with no quotes. A changed setting takes effect only in a new container: run docker stop sequences && docker rm sequences, then your docker run command again, with the new -e value or the edited env file. docker restart keeps the old settings.
| Setting | Default in the image | What it does |
|---|---|---|
BIOSEQUENCE_LICENSE_FILE | /config/license.json | The license file |
BIOSEQUENCE_DATA_DIR | /data | Snapshots, jobs and results. Keep it on the sequences-data volume |
BIOSEQUENCE_PORT | 8765 | The port the service listens on, 1024 to 65535 |
BIOSEQUENCE_HOSTED_BIND | 0.0.0.0 | The address inside the container with sign-in on. With sign-in off the service uses 127.0.0.1 whatever this says |
BIOSEQUENCE_ALLOWED_HOSTS | none | The host name people use, such as sequences.example.org. Setting it turns sign-in on. Separate several names with commas |
BIOSEQUENCE_SITE_URL | https:// and the first allowed host | The address people sign in at: https:// and one of the allowed hosts, with no port or path. Its /api/auth/callback is the redirect URI you register with your provider |
BIOSEQUENCE_OIDC_ISSUER | none | Your provider's issuer URL |
BIOSEQUENCE_OIDC_CLIENT_ID | none | The client ID of the Sequences application at your provider |
BIOSEQUENCE_OIDC_CLIENT_SECRET_FILE | none | A file that holds the client secret, such as /config/oidc-client-secret. BIOSEQUENCE_OIDC_CLIENT_SECRET takes the secret itself instead |
BIOSEQUENCE_OIDC_CLIENT_AUTH | client_secret_basic | How the service sends the secret: client_secret_basic or client_secret_post |
BIOSEQUENCE_OIDC_NAME | your organization's account | Shown on the sign-in button: "Sign in with your organization's account" |
BIOSEQUENCE_OIDC_SCOPES | openid email profile | Scopes the service asks for. Must include openid |
BIOSEQUENCE_OIDC_USER_CLAIM | email | The claim that names a person. With email, the provider must also send email_verified as true |
BIOSEQUENCE_OIDC_GROUP_CLAIM | groups | The claim that lists a person's groups or roles |
BIOSEQUENCE_OIDC_REQUIRED_GROUP | none | Only people with this value in the group claim may sign in |
BIOSEQUENCE_ALLOWED_DOMAINS | none | Domains that may sign in, such as example.org, separated by commas: people whose user claim ends in @ and one of them |
BIOSEQUENCE_INVITE_FILE | none | A file with one user per line (the user claim's value), where # starts a comment |
BIOSEQUENCE_ADMINS | none | People who see the usage page at /admin and make connector tokens, as the user claim names them, separated by commas. They must still be allowed to sign in |
BIOSEQUENCE_SESSION_HOURS | 12 | How long a sign-in lasts, 1 to 168 hours |
BIOSEQUENCE_SESSION_SECRET_FILE | /data/session-secret, made on first start | The key that signs session cookies, 32 characters or more |
BIOSEQUENCE_TOKEN_DAYS | 90 | The longest an API token may last, 1 to 90 days. 0 turns API tokens off. See API tokens for programs |
BIOSEQUENCE_THREADS | CPU count minus 2, at most 14 | Threads for one search, 1 to 14 |
BIOSEQUENCE_CONCURRENCY | 1 | How many searches of the snapshot, exact alignments and portfolio exports run at once, 1 to 16. The others wait their turn. A coordinator takes 0 to 16. See Settings for several servers |
BIOSEQUENCE_MAX_JOBS | 8 | How many searches the queue holds at once, waiting and running, 1 to 256. A case's run uses the free places and leaves 2 for searches people start from the page |
BIOSEQUENCE_JOB_SECONDS | 3600 | Time limit for one search or portfolio export, 1 to 86,400 seconds |
BIOSEQUENCE_CPU_HOURS_PER_DAY | none | The CPU hours each account's searches may use in a UTC day, a number above 0 and up to 100,000, such as 24. At the limit, the account's new searches are refused until 00:00 UTC. Unset, there is no limit. See Compute limits |
BIOSEQUENCE_CONFIRM_MINUTES | none | A search whose estimate is longer, 1 to 1,440 minutes, asks the person before it queues. Unset, no search asks for its time. See Compute limits |
BIOSEQUENCE_LARGE_SETS | none | Sets whose searches always ask before they queue, separated by commas: a snapshot set by its name, such as trembl, or an attached database by the path it was attached from or by its ID |
BIOSEQUENCE_MAX_OUTPUT_BYTES | 8 GiB | Largest result file of one search: a whole number of bytes, with no unit, from 1 KiB to 4 TiB |
BIOSEQUENCE_LISTING_BYTES | 256 MiB | Largest sequence listing file people can upload: a whole number of bytes, with no unit, from 1 MiB to 64 GiB. Keep your proxy's upload limit above it |
BIOSEQUENCE_PATENT_LINK | https://patents.google.com/patent/{office}{number}{kind} | The link on each patent's number in the results. The default writes each number as Google Patents writes that office's numbers in that era, and links a publication Google Patents does not have to Espacenet. Another address is filled in as you give it. An empty value removes the links |
BIOSEQUENCE_CONTEXT | none | A patent context folder made with the context command, such as /data/context/<snapshotId>-1. With it, a US publication's pane quotes what the patent says about the matching sequences. See Patent context |
BIOSEQUENCE_MAIL_FROM | none | The sender of watch emails, such as Sequences <noreply@example.org>. Set it with BIOSEQUENCE_RESEND_KEY, or neither. See Watch emails |
BIOSEQUENCE_RESEND_KEY | none | The API key of your account at Resend, the email delivery service that sends watch emails. Without both mail settings, watches show what they find on the page only |
BIOSEQUENCE_MAIL_DETAILS | off | on makes each watch email name the watch and list its new matches or publications. Unset or off, an email says only how many new matches or publications one of the person's watches found, and names no watch, query or match. Any other value stops the service at start. See Watch emails |
BIOSEQUENCE_CLAIMS_MODEL_URL | none | The address a case's claims go to when someone asks a language model for features the rules missed, and a text's facts when someone asks it for a draft (Pilot), with the whole path, such as http://127.0.0.1:8000/v1/chat/completions. Unset, suggestions are off. See Claim suggestions from a model and Network traffic |
BIOSEQUENCE_CLAIMS_MODEL_API | sequences | The format of the call. openai-chat posts a chat completions call, the format vLLM, llama.cpp's server and Ollama take: the instructions as the system message and the claims as JSON in the user message, at temperature 0, asking for JSON back. sequences posts the service's own JSON request and reads a JSON answer, for a service of your own in front of a model; for that format, write to support@conductscience.com |
BIOSEQUENCE_CLAIMS_MODEL | none | The model name the service puts in each request, up to 200 characters. Needed with the address |
BIOSEQUENCE_CLAIMS_MODEL_HOSTS | none | Host names the address may use besides 127.0.0.1 and localhost, separated by commas. Such a host must use https:// |
BIOSEQUENCE_CLAIMS_MODEL_KEY | none | A key the service sends to the model as Authorization: Bearer, if the model needs one. It is never written to the log or to a case |
BIOSEQUENCE_CLAIMS_MODEL_SECONDS | 900 with openai-chat, 120 otherwise | How long the service waits for the model, 5 to 3,600 seconds. A model on processors alone can take minutes for one case |
BIOSEQUENCE_PAPERS | none | A paper folder made with the papers command, such as /data/papers/<id>. With it, each paper a hit cites reads as its PubMed citation, dated to the day it first came out, with its marks. See Paper citations and marks |
BIOSEQUENCE_CROSSREF_MAILTO | none | A contact address for the crossref command to send Crossref with its client name, as Crossref asks of automated clients. Without it the command sends no address. See Dates from Crossref |
BIOSEQUENCE_TAXA | none | A taxa folder made with the taxa command, such as /data/taxa/<id>. With it, each record of a hit has its NCBI taxon and species group, and a search's hits can be filtered by species or group. See Species of each hit |
BIOSEQUENCE_ROR | none | A ROR name index made with the ror command, such as /data/ror/v2.15-2026-10-20. With it, the Applicants tab suggests the ROR record of an applicant whose name matches. See ROR suggestions |
BIOSEQUENCE_ANTIBODY_NAMES | none | An antibody name table made with the antibody-names commands, such as /data/antibody-names/chembl_37. With it, antibody CDR matches name the therapeutic antibodies they share sequence with, from ChEMBL. See Antibody names |
BIOSEQUENCE_ADDONS | none | The add-ons to run, separated by commas: clinical for clinical context, funding for the funding trail, variants for known variants, drugs for drug names, genes for the human genes of Swiss-Prot and patent protein hits, claimed for claimed sequences, families for patent families, status for patent status and modifications for modified residues and bases. Empty or none runs none. An add-on name this version does not know stops the service at start. See Clinical context, Funding trail, Known variants, Drug names, Claimed sequences, Patent families, Patent status and Modifications |
BIOSEQUENCE_CLINICAL | none | The clinical context folder that load made, such as /data/clinical/<release>. The service does not start when BIOSEQUENCE_ADDONS lists clinical and this is unset, or when this is set and clinical is not listed |
BIOSEQUENCE_FUNDING | none | The funding trail folder that load made, such as /data/funding/<release>. The service does not start when BIOSEQUENCE_ADDONS lists funding and this is unset, or when this is set and funding is not listed |
BIOSEQUENCE_VARIANTS | none | The known variants folder that load made, such as /data/variants/<release>. The service does not start when BIOSEQUENCE_ADDONS lists variants and this is unset, or when this is set and variants is not listed |
BIOSEQUENCE_DRUGS | none | The drug names folder that load made, such as /data/drugs/<release>. The service does not start when BIOSEQUENCE_ADDONS lists drugs and this is unset, or when this is set and drugs is not listed |
BIOSEQUENCE_GENES | none | The genes folder that load made, such as /data/genes/<release>. The service does not start when BIOSEQUENCE_ADDONS lists genes and this is unset, or when this is set and genes is not listed |
BIOSEQUENCE_CLAIMED | none | The claimed sequences folder, mounted read-only, such as /claimed. The service does not start when BIOSEQUENCE_ADDONS lists claimed and this is unset, or when this is set and claimed is not listed |
BIOSEQUENCE_FAMILIES | none | The patent families folder, mounted read-only, such as /families. The service does not start when BIOSEQUENCE_ADDONS lists families and this is unset, or when this is set and families is not listed |
BIOSEQUENCE_STATUS | none | The patent status folder, mounted read-only, such as /status. The service does not start when BIOSEQUENCE_ADDONS lists status and this is unset, or when this is set and status is not listed |
BIOSEQUENCE_MODIFICATIONS | none | The modifications folder, mounted read-only, such as /modifications. The service does not start when BIOSEQUENCE_ADDONS lists modifications and this is unset, or when this is set and modifications is not listed |
BIOSEQUENCE_ROLE | single | coordinator or worker when several servers share one queue. See Several servers |
BIOSEQUENCE_WORKER_KEY_FILE | none | A file that holds the worker key, which every server of several shares. BIOSEQUENCE_WORKER_KEY takes the key itself instead |
BIOSEQUENCE_WORKER_BIND, BIOSEQUENCE_WORKER_PORT | 127.0.0.1, 8766 | Where a coordinator's worker port listens. BIOSEQUENCE_WORKER_TLS_CERT and BIOSEQUENCE_WORKER_TLS_KEY turn on TLS there |
BIOSEQUENCE_COORDINATOR_URL, BIOSEQUENCE_WORKER_NAME | none, the container's ID | A worker's coordinator and its own name. See Settings for several servers |
BIOSEQUENCE_EXPORT_FORMATS_FILE | none | A file of export formats, such as /config/export-formats.json: for each export of rows, the columns, headers, file type and redactions. A file that does not fit stops the service at start. See Export formats |
BIOSEQUENCE_CONFIDENTIAL_SETS | none | A comma-separated list of what holds confidential data: snapshot sets by name, such as patent_nucl, sources of a snapshot's records by ID, such as ebi-epo, and attached databases by the path they were attached from or by ID. Every external export format leaves out their rows. See Confidential sets |
BIOSEQUENCE_RETENTION_DAYS | none | How many days records are kept, 1 to 36,500: searches that no case holds, portfolio exports and finished batches are removed once they are that old. Unset, records stay until their owners delete them. See How long records are kept |
NODE_EXTRA_CA_CERTS | none | A PEM file of your own certificate authority, if your provider's certificate, or on a worker the coordinator's, comes from one |
With sign-in on, the service needs at least one rule for who may use it: BIOSEQUENCE_INVITE_FILE, BIOSEQUENCE_ALLOWED_DOMAINS or BIOSEQUENCE_OIDC_REQUIRED_GROUP. Write the settings says how the rules combine. A setting that is missing or does not fit stops the service at start with a message that names it.
Commands
Run a command as docker run --rm -v sequences-data:/data conductscience/sequences:<version> <command>. check-license also needs -v /etc/sequences:/config:ro, and load the import folder, as shown above. Only serve and check-license read the license file, and a worker's serve reads none. A snapshot's ID is the name of its folder under corpora/, and this lists them:
docker run --rm -v sequences-data:/data --entrypoint ls conductscience/sequences:<version> /data/corpora
| Command | What it does |
|---|---|
serve | Start the service (the default command) |
versions | Show the release and the version of every search program |
check-license | Show what the license file allows |
fips-check | In the FIPS image, run each algorithm the service uses and say whether it ran in the FIPS provider. Exit 1 if a check fails or FIPS mode is off, as it is in the standard image |
healthcheck | Exit with 0 if the service in the same container answers on 127.0.0.1. The image's health check runs it inside the running container, and Health and logs shows how to read the result. After 3 failures in a row it stops the service, so that the container's restart rule starts it again |
drain [--wait <minutes>] | On a worker, finish the searches it runs and take no more, then exit with 0. Exit 1 if the worker starts again first, or after 60 minutes or the minutes --wait gives (1 to 1440). Run it in the running worker with docker exec. See Drain and stop a worker |
load <folder> --sums <file> | Load a snapshot, delta, clinical context, funding trail or drug names copy, checking every file |
verify <snapshotId> [--delta <deltaId>] | Run a snapshot's or delta's checks again |
sums <snapshotId> [--delta <deltaId>] | Print the SHA-256 list of a snapshot or delta folder. sums --folder <folder> prints any folder's, such as a clinical context, funding trail or drug names folder |
attribution <snapshotId> [--delta <deltaId>] | Print the sources and terms of a snapshot or delta |
fetch, build, delta, deltas | Download the public source files, build a snapshot, add newer files as a delta, and list deltas |
papers --out <folder> --snapshot <folder> --pubmed <folder> ... | Make a paper folder from PubMed's files and the Retraction Watch Database. See Paper citations and marks |
crossref --answers <folder> --snapshot <folder> ... | Ask Crossref for the dates of the cited papers PubMed does not hold, for papers --crossref. See Dates from Crossref |
taxa-scan --inputs <folder> --source all --out <folder> | Read the NCBI taxon each record states, from the files a snapshot was built from, into a folder for each source. See Species of each hit |
taxa --taxdump <file> --out <folder> [--scans <folder>]... | Make a taxa folder from NCBI's taxonomy and the source scans. See Species of each hit |
ror --zip <file> --out <folder> | Make the name index for ROR suggestions from ROR's data zip. See ROR suggestions |
antibody-names fetch --release <n> --out <folder>, antibody-names build --from <folder> --out <folder>, antibody-names check --table <folder> | Download a ChEMBL release's biotherapeutic sequences and names, make the antibody name table from them, and check a table before you set it. See Antibody names |
sample add <search ID> --name <name> --title <words> --source <words> --source-url <https link> | Make a sample result from a finished search. See Sample results |
sample list, sample remove <name> | List the sample results, or remove one |
audit-export --day <YYYY-MM-DD>, or --from <time> --to <time> | Write the audit log and the case logs for a UTC day or a time range as JSON lines for a SIEM, reading the data volume only. See Audit export for a SIEM |
context extract, context assemble, context check | Make a patent context folder from the US patent office's weekly full-text files, and check it. See Patent context |
context ops | Add the titles and English abstracts of other offices' publications from EPO's Open Patent Services, only with BIOSEQUENCE_OPS=on. See Other offices |
help | List every command and its options |
Health and logs
The image checks itself every 30 seconds, after a 90-second start period, by asking the service on 127.0.0.1 inside the container. To see the result, run:
docker inspect --format '{{.State.Health.Status}}' sequences
healthy means the service answers. At start the service may wait up to 50 seconds for the lock (.lock) that a crashed container left on the data volume to expire. See Upgrades.
Docker restarts a container only when it exits, not when it fails its health check, so the check does the rest. After the start period, when it has failed 3 times in a row, about 90 seconds, and Docker marks the container unhealthy, the check stops the service. The container exits, and its restart rule starts it again: --restart unless-stopped in this guide's commands, or the AWS kit's service unit, which starts it 10 seconds after it stops. The log says:
The health check failed 3 times in a row, so it stopped the service (SIGTERM). The container's restart rule starts it again.
A service that still fails the next check is ended (SIGKILL). A stop by the check is like any other stop for the searches the server runs. A worker gives its searches back to the queue, or, if it was ended, they go back a minute later, counted as cut off once; when it starts again it joins its coordinator, as at any start. A container started without a restart rule, such as one run with docker run --rm, stays stopped. A one-off container of the image, such as one that runs load, audit-export or a backup's tar, runs no service, so there the check only reports and stops nothing.
Measured on release 0.92.0: a service frozen in the middle of its searches, so that it stopped answering while its container still ran, was started again by the check and its restart rule 142 seconds after it froze on a worker, and 111 seconds on a coordinator. Every search of the batch completed once.
A worker opens no port, so its health check reads the state the worker writes to its data volume every 10 seconds instead. It is healthy while the worker writes it and its coordinator has not refused it. A worker that cannot reach its coordinator stays healthy, says so in its log and tries again every 20 seconds. See Several servers.
For an uptime monitor outside the container, read https://sequences.example.org/api/health with sign-in on, or http://127.0.0.1:8765/api/health on the machine itself. It needs no sign-in. It answers 200 with {"ok":true} when BLAST+ answers and a snapshot is installed, and 503 with {"ok":false} otherwise. It says nothing else, and it asks the search tools at most every 30 seconds however often it is read. With sign-in on, it also answers at the server's IP address, for a load balancer's health check, which cannot name your host. Nothing else answers there.
The service writes its log to standard output and standard error, so docker logs sequences shows it:
- At start: the host names and issuer (sign-in on) or the local address (sign-in off), the license summary, the data folder, the paper folder if you set one, whether watch emails are on and with or without details, the ROR release that applicant suggestions come from, or that they are off, the add-ons that run (
Add-ons: none.when none do), and the records it skipped because it cannot read them. - Sign-in problems, such as a provider that cannot be reached, a token that fails a check, or an account that is refused, with the reason.
- A license file or invite list that cannot be read.
- Each API token made or revoked, with the account and the token's name, never the token itself.
- On a coordinator: the worker port; each worker that joins, drains, leaves, is lost or is refused; and each search that waits in the queue again, or fails, because its worker stopped answering or sent a result that failed a check. On a worker: its coordinator, whether its copies match, and each search it runs, with its hits and time.
A search's job.json, a database's manifest.json or the run-time estimates (calibration.json) that cannot be read, such as one a full disk cut off, does not stop the start or empty a list. The service skips the file and leaves it on disk as it is: that search or database is not listed, and the estimates start from their first values and do not write over the file. The start log names each one, such as Skipped 1 record that cannot be read. Each stays on disk in the data folder: jobs/<id>/job.json. To get one back, restore it from a backup (see Backups): a database shows again at once, a search and the estimates at the next start.
Sign-ins, API tokens, starts, invite file changes and a coordinator's workers also go into the service's audit log on the data volume, which keeps them. See Audit export for a SIEM.
Queries, sequences and results are never written to the log. Docker keeps logs without a size limit unless you set one. These options to docker run keep five files of 10 MB:
--log-opt max-size=10m --log-opt max-file=5
Usage
With sign-in on, the people named in BIOSEQUENCE_ADMINS find Usage under their account button. The usage page, /admin, counts searches and other work (alignments, imports and portfolio exports) for each account and day over the last 7, 30 or 90 days. A day is the UTC date a job started. The page names each account as the user claim named it at its last sign-in, which the service keeps in accounts.json on the data volume. For anyone else, the page does not exist. On a server that runs under a license, the page starts with the license's plan and last day, and with its message when the end is near or new work has stopped. See The license file.
The page also adds up what searches used to run, as each search records it (see What each search uses): the CPU time in all, the CPU time and data read for each account, and for each day the CPU time, run time, peak memory and data read by account and by set. Over the tables it states the server's compute limits. Download CSV saves one row for each day, account and set, with the columns day, account, account_ref, set, searches, cpu_seconds, wall_seconds, peak_memory_bytes and bytes_read. A program gets the same file from GET /api/admin/usage.csv?days=30, with 1 to 90 days.
The usage page also lists every API token: whose it is, its name, when it was made, last used and expires, and whether it is active, expired or revoked, and by whom. An admin can revoke any token there, and the person who made it then reads "Revoked by an admin". Searches a program starts with a token count for the token's account, as any search does.
An admin can also download a file of the decisions in every case on the server, to measure how well the claim rules and model prompts work: GET /api/admin/decisions.csv, or GET /api/admin/decisions.json, which adds totals. Each row is a claim feature, a call to the claims model, a dropped suggestion, or a decision on a drafted sentence. A row gives the case ID, the account by the first 8 characters of its ID (the ref that GET /api/admin/usage gives), the claims rules version, the feature's kind, rule, source and status and whether it was edited, the model and the prompt's ID and SHA-256, and a drop reason in fixed words. The file holds no claim words, quotes, notes or edits, since claims pasted into a case can be from an application that is not yet published. A case's own decisions, with its words, download from the case's Report step. See The decisions file.
On a coordinator, the page ends with its workers. See The Servers section.
Admins can also export the service's audit log with every case's log. See Audit export for a SIEM.
Sizing and cost
Each search records what it used to run, so you can see what your searches cost and set limits on them. The last four parts of this section help you choose a server: searches measured on the hosted service, which runs the same release and programs as your own server; the searches an hour of a load test and of steady loads of four kinds of search, with the hosted service's patent snapshot and limits; the size on disk of its snapshots and of NCBI core_nt; and the AWS prices of the server types the AWS kit offers. The kit's default data volume, 200 GiB, holds the patent snapshot; to add the snapshot of Swiss-Prot, TrEMBL and ESTs, choose a larger one.
What each search uses
A search keeps four measures in its job (compute in GET /api/jobs/{id}) and in its search record, under the record's SHA-256:
| Measure | What it counts |
|---|---|
CPU time, cpuSeconds | The processor time of the programs the search ran, on all their threads: BLAST+, FASTA36, or the threads that read the set for an exact, pattern or antibody search |
Run time, wallSeconds | The time from the search's start to its end |
Peak memory, peakMemoryBytes | The most memory one of its programs held. The threads of a scan count together |
Data read, bytesRead | The size of the set files its programs read: a BLAST database's sequence and index files, or the set's sequences for FASTA36 and the scans |
The service's own work for a search, such as writing its rows and joining publications, is not counted. A search that fails or is cancelled keeps what it used until then. On several servers, the worker that runs a search measures it and sends the measures with the result. The container reads CPU time and peak memory from Linux. On the local workbench under Windows they read "Not recorded", as they do for searches made before release 0.87.0.
BLAST maps a database's files into memory, so its peak memory counts the pages of the database it read, which Linux can give back to other programs. A BLAST search of NCBI core_nt peaked at 51.6 to 53.5 GiB, near the container's 56 GiB, on November 14, 2026, and at 11.8 GiB on November 20.
Compute limits
With none of these settings, searches run as they come. Each takes effect in a new container, as any setting does, and the start's log line states them.
BIOSEQUENCE_CPU_HOURS_PER_DAYcaps the CPU time each account's searches use in a UTC day. Once the searches an account started that day have recorded that much, its new searches are refused with "Your searches have used today's limit of 24 CPU hours. New searches can start at 00:00 UTC, in 3 hours 12 minutes." A program gets 429, withRetry-Aftergiving the seconds until 00:00 UTC. Searches already queued or running go on, and their time counts when they end. The limit covers every way a search starts: the search form, a combined search, Run again, a batch, an antibody search, a listing's searches and a case's run.BIOSEQUENCE_CONFIRM_MINUTESmakes a search whose estimate is longer ask before it queues. The page asks "This search should take about 25 minutes. This server asks before a search over 15 minutes. Run it?", and the search starts only if the person chooses OK. A program gets 409 withconfirm, which gives the estimate, and sends the same request again with"confirmed": true. Searches made together, as a combined search or a listing's searches, ask once for their time together.BIOSEQUENCE_LARGE_SETSnames sets whose searches always ask, with their estimate, such asBIOSEQUENCE_LARGE_SETS=trembl,est. An antibody search has no estimate, so it asks only when it reads one of these sets.
A case's run does not ask: TrEMBL and attached databases join a plan only when someone adds them, with the time they should take, and locking the plan decides it. The usage page and its CSV show each account's CPU time, which the daily limit counts.
Measured searches
Measured on releases 0.87.1, 0.90.0 and 0.111.2, in 4 runs that started between November 14, 2026, 19:23 UTC and November 20, 2026, 21:59 UTC, on a server with 12 CPUs and 56 GiB of memory for the container, one search at a time. CPU time and run time are in seconds.
The CPU time of a search on average, by method and set, over the queries in the tables after it:
| Method | Patent DNA | Patent proteins | Swiss-Prot | NCBI core_nt | TrEMBL | ESTs |
|---|---|---|---|---|---|---|
| Automatic | 91.0 s | 74.4 s | 4.0 s | Not offered | 1,261 s | 66.9 s |
| Alignment (BLAST) | 61.9 s | 21.5 s | 1.0 s | 910 s | 307 s | 45.2 s |
| Smith-Waterman | 1,087 s | 74.2 s | 4.1 s | Not offered | 1,263 s | 950 s |
| Global on the query | 1,219 s | 61.5 s | 3.9 s | Not offered | 1,212 s | 1,109 s |
| Global on both sequences | 62.4 s | 14.2 s | 0.6 s | Not offered | 180 s | 25.8 s |
| Exact text | 113 s | 4.9 s | 0.4 s | Not offered | 95.5 s | 74.3 s |
| Pattern | 118 s | 8.7 s | 0.6 s | Not offered | 147 s | 86.4 s |
| Antibody regions | Not offered | 95.1 s | 5.0 s | Not offered | 1,623 s | Not offered |
Patent DNA, Main plus New (snapshot 2026-11-13-5e447c75c43f and addition 2026-11-17-c7f686506ac1, 76,205,756 sequences of 35,101,562,749 bases):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Automatic | 30 bases | 156 s | 18.5 s | 10.0 GiB | 43.1 GiB |
| Automatic | 309 bases | 38.3 s | 5.8 s | 9.0 GiB | 9.2 GiB |
| Automatic | 1,509 bases | 79.0 s | 9.9 s | 9.7 GiB | 9.2 GiB |
| Alignment (BLAST) | 30 bases | 73.9 s | 10.7 s | 10.0 GiB | 9.2 GiB |
| Alignment (BLAST) | 309 bases | 34.1 s | 5.5 s | 9.0 GiB | 9.2 GiB |
| Alignment (BLAST) | 1,509 bases | 77.7 s | 9.8 s | 9.7 GiB | 9.2 GiB |
| Smith-Waterman | 21 bases | 968 s | 81.6 s | 6.4 GiB | 33.9 GiB |
| Smith-Waterman | 30 bases | 943 s | 79.5 s | 6.4 GiB | 33.9 GiB |
| Smith-Waterman | 50 bases | 1,350 s | 113 s | 6.4 GiB | 33.9 GiB |
| Global on the query | 21 bases | 1,183 s | 145 s | 7.1 GiB | 33.9 GiB |
| Global on the query | 30 bases | 1,182 s | 138 s | 7.1 GiB | 33.9 GiB |
| Global on the query | 50 bases | 1,293 s | 109 s | 6.4 GiB | 33.9 GiB |
| Global on both sequences | 21 bases | 69.5 s | 47.7 s | 6.4 GiB | 33.9 GiB |
| Global on both sequences | 30 bases | 61.0 s | 46.8 s | 6.4 GiB | 33.9 GiB |
| Global on both sequences | 50 bases | 56.7 s | 47.1 s | 6.4 GiB | 33.9 GiB |
| Exact text | 30 bases | 87.7 s | 8.0 s | 3.1 GiB | 33.9 GiB |
| Exact text | 309 bases | 137 s | 12.2 s | 3.1 GiB | 33.9 GiB |
| Exact text | 1,509 bases | 115 s | 10.6 s | 3.1 GiB | 33.9 GiB |
| Pattern | 30 bases | 124 s | 12.3 s | 12.3 GiB | 33.9 GiB |
| Pattern | 309 bases | 111 s | 10.6 s | 12.4 GiB | 33.9 GiB |
| Pattern | 1,509 bases | 118 s | 10.9 s | 12.4 GiB | 33.9 GiB |
Patent proteins, Main plus New (snapshot 2026-11-13-5e447c75c43f and addition 2026-11-17-c7f686506ac1, 22,941,706 sequences of 2,657,604,038 residues):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Automatic | 16 residues | 26.9 s | 10.3 s | 5.0 GiB | 2.8 GiB |
| Automatic | 71 residues | 60.9 s | 10.8 s | 5.1 GiB | 2.8 GiB |
| Automatic | 258 residues | 135 s | 15.9 s | 5.1 GiB | 2.8 GiB |
| Alignment (BLAST) | 16 residues | 10.4 s | 3.6 s | 2.8 GiB | 2.7 GiB |
| Alignment (BLAST) | 71 residues | 14.9 s | 1.8 s | 2.7 GiB | 2.7 GiB |
| Alignment (BLAST) | 258 residues | 39.1 s | 6.5 s | 3.1 GiB | 2.7 GiB |
| Smith-Waterman | 16 residues | 26.0 s | 8.8 s | 5.1 GiB | 2.8 GiB |
| Smith-Waterman | 71 residues | 60.6 s | 10.4 s | 5.1 GiB | 2.8 GiB |
| Smith-Waterman | 258 residues | 136 s | 14.4 s | 5.1 GiB | 2.8 GiB |
| Global on the query | 16 residues | 39.3 s | 9.0 s | 5.1 GiB | 2.8 GiB |
| Global on the query | 71 residues | 55.0 s | 9.9 s | 5.1 GiB | 2.8 GiB |
| Global on the query | 258 residues | 90.1 s | 10.8 s | 5.1 GiB | 2.8 GiB |
| Global on both sequences | 16 residues | 9.5 s | 11.3 s | 5.1 GiB | 2.8 GiB |
| Global on both sequences | 71 residues | 9.8 s | 9.4 s | 5.1 GiB | 2.8 GiB |
| Global on both sequences | 258 residues | 23.2 s | 8.9 s | 5.0 GiB | 2.8 GiB |
| Exact text | 16 residues | 5.0 s | 0.6 s | 2.9 GiB | 2.8 GiB |
| Exact text | 71 residues | 4.6 s | 0.6 s | 2.9 GiB | 2.8 GiB |
| Exact text | 258 residues | 5.0 s | 0.6 s | 2.9 GiB | 2.8 GiB |
| Pattern | 16 residues | 8.5 s | 1.0 s | 5.7 GiB | 2.8 GiB |
| Pattern | 71 residues | 9.2 s | 1.0 s | 5.7 GiB | 2.8 GiB |
| Pattern | 258 residues | 8.4 s | 1.0 s | 5.7 GiB | 2.8 GiB |
| Antibody regions | 120 residues, heavy chain | 79.1 s | 21.0 s | 5.2 GiB | 5.7 GiB |
| Antibody regions | 107 residues, light chain | 68.9 s | 21.2 s | 5.2 GiB | 5.7 GiB |
| Antibody regions | 227 residues, both chains | 137 s | 28.2 s | 5.2 GiB | 5.7 GiB |
Swiss-Prot (snapshot 2026-11-10-37cac4ca8f21, 487,796 sequences of 186,172,811 residues):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Automatic | 16 residues | 1.3 s | 0.3 s | 140.8 MiB | 184.4 MiB |
| Automatic | 71 residues | 3.6 s | 0.4 s | 274.9 MiB | 184.4 MiB |
| Automatic | 258 residues | 7.3 s | 0.7 s | 274.7 MiB | 184.4 MiB |
| Alignment (BLAST) | 16 residues | 0.5 s | 0.2 s | 235.5 MiB | 181.7 MiB |
| Alignment (BLAST) | 71 residues | 0.6 s | 0.1 s | 205.8 MiB | 181.7 MiB |
| Alignment (BLAST) | 258 residues | 1.8 s | 0.2 s | 172.2 MiB | 181.7 MiB |
| Smith-Waterman | 16 residues | 1.3 s | 0.3 s | 204.2 MiB | 184.4 MiB |
| Smith-Waterman | 71 residues | 3.5 s | 0.4 s | 274.4 MiB | 184.4 MiB |
| Smith-Waterman | 258 residues | 7.3 s | 0.7 s | 274.7 MiB | 184.4 MiB |
| Global on the query | 16 residues | 2.5 s | 0.6 s | 214.0 MiB | 184.4 MiB |
| Global on the query | 71 residues | 3.5 s | 0.5 s | 284.9 MiB | 184.4 MiB |
| Global on the query | 258 residues | 5.7 s | 0.5 s | 228.0 MiB | 184.4 MiB |
| Global on both sequences | 16 residues | 0.2 s | 0.2 s | 222.8 MiB | 184.4 MiB |
| Global on both sequences | 71 residues | 0.3 s | 0.3 s | 219.7 MiB | 184.4 MiB |
| Global on both sequences | 258 residues | 1.4 s | 0.2 s | 225.4 MiB | 184.4 MiB |
| Exact text | 16 residues | 0.4 s | 0.1 s | 297.5 MiB | 184.4 MiB |
| Exact text | 71 residues | 0.4 s | 0.1 s | 297.5 MiB | 184.4 MiB |
| Exact text | 258 residues | 0.4 s | 0.1 s | 297.5 MiB | 184.4 MiB |
| Pattern | 16 residues | 0.6 s | 0.1 s | 482.0 MiB | 184.4 MiB |
| Pattern | 71 residues | 0.6 s | 0.1 s | 481.9 MiB | 184.4 MiB |
| Pattern | 258 residues | 0.6 s | 0.1 s | 482.1 MiB | 184.4 MiB |
| Antibody regions | 120 residues, heavy chain | 4.2 s | 0.8 s | 297.6 MiB | 368.8 MiB |
| Antibody regions | 107 residues, light chain | 3.6 s | 0.7 s | 297.6 MiB | 368.8 MiB |
| Antibody regions | 227 residues, both chains | 7.2 s | 1.3 s | 297.6 MiB | 368.8 MiB |
NCBI core_nt (attached BLAST database, 131,880,465 sequences of 1,089,667,566,791 bases):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Alignment (BLAST) | 30 bases | 910 s | 372 s | 11.8 GiB | 257.1 GiB |
TrEMBL (snapshot 2026-11-10-37cac4ca8f21, 147,445,217 sequences of 58,101,597,141 residues):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Automatic | 16 residues | 374 s | 91.0 s | 19.9 GiB | 56.5 GiB |
| Automatic | 71 residues | 1,075 s | 101 s | 19.9 GiB | 56.5 GiB |
| Automatic | 258 residues | 2,336 s | 197 s | 19.9 GiB | 56.5 GiB |
| Alignment (BLAST) | 16 residues | 130 s | 15.3 s | 38.3 GiB | 55.3 GiB |
| Alignment (BLAST) | 71 residues | 224 s | 20.0 s | 40.0 GiB | 55.3 GiB |
| Alignment (BLAST) | 258 residues | 568 s | 54.3 s | 54.7 GiB | 55.3 GiB |
| Smith-Waterman | 16 residues | 377 s | 93.1 s | 19.9 GiB | 56.5 GiB |
| Smith-Waterman | 71 residues | 1,075 s | 99.7 s | 19.9 GiB | 56.5 GiB |
| Smith-Waterman | 258 residues | 2,338 s | 197 s | 19.9 GiB | 56.5 GiB |
| Global on the query | 16 residues | 732 s | 97.6 s | 19.9 GiB | 56.5 GiB |
| Global on the query | 71 residues | 1,076 s | 101 s | 19.9 GiB | 56.5 GiB |
| Global on the query | 258 residues | 1,827 s | 154 s | 19.9 GiB | 56.5 GiB |
| Global on both sequences | 16 residues | 54.3 s | 94.9 s | 19.9 GiB | 56.5 GiB |
| Global on both sequences | 71 residues | 73.0 s | 97.5 s | 19.9 GiB | 56.5 GiB |
| Global on both sequences | 258 residues | 413 s | 96.6 s | 19.9 GiB | 56.5 GiB |
| Exact text | 16 residues | 115 s | 11.2 s | 3.1 GiB | 56.5 GiB |
| Exact text | 71 residues | 104 s | 10.7 s | 3.1 GiB | 56.5 GiB |
| Exact text | 258 residues | 68.3 s | 7.9 s | 3.1 GiB | 56.5 GiB |
| Pattern | 16 residues | 144 s | 15.5 s | 15.0 GiB | 56.5 GiB |
| Pattern | 71 residues | 148 s | 16.3 s | 14.2 GiB | 56.5 GiB |
| Pattern | 258 residues | 149 s | 15.9 s | 15.1 GiB | 56.5 GiB |
| Antibody regions | 120 residues, heavy chain | 1,316 s | 257 s | 19.9 GiB | 113.0 GiB |
| Antibody regions | 107 residues, light chain | 1,155 s | 280 s | 19.9 GiB | 113.0 GiB |
| Antibody regions | 227 residues, both chains | 2,397 s | 554 s | 19.9 GiB | 113.0 GiB |
ESTs (snapshot 2026-11-10-37cac4ca8f21, 67,045,879 sequences of 37,304,066,442 bases):
| Method | Query | CPU time | Run time | Peak memory | Data read |
|---|---|---|---|---|---|
| Automatic | 30 bases | 136 s | 26.9 s | 10.2 GiB | 45.5 GiB |
| Automatic | 309 bases | 15.0 s | 13.8 s | 9.5 GiB | 9.7 GiB |
| Automatic | 1,509 bases | 50.0 s | 9.0 s | 10.3 GiB | 9.7 GiB |
| Alignment (BLAST) | 30 bases | 46.8 s | 6.0 s | 10.2 GiB | 9.7 GiB |
| Alignment (BLAST) | 309 bases | 37.8 s | 3.9 s | 9.5 GiB | 9.7 GiB |
| Alignment (BLAST) | 1,509 bases | 51.1 s | 5.0 s | 10.3 GiB | 9.7 GiB |
| Smith-Waterman | 21 bases | 842 s | 70.7 s | 9.1 GiB | 35.8 GiB |
| Smith-Waterman | 30 bases | 814 s | 68.3 s | 9.1 GiB | 35.8 GiB |
| Smith-Waterman | 50 bases | 1,195 s | 99.9 s | 9.1 GiB | 35.8 GiB |
| Global on the query | 21 bases | 1,052 s | 89.2 s | 9.2 GiB | 35.8 GiB |
| Global on the query | 30 bases | 1,031 s | 87.4 s | 9.1 GiB | 35.8 GiB |
| Global on the query | 50 bases | 1,245 s | 104 s | 9.1 GiB | 35.8 GiB |
| Global on both sequences | 21 bases | 26.4 s | 25.1 s | 9.1 GiB | 35.8 GiB |
| Global on both sequences | 30 bases | 25.9 s | 24.6 s | 9.1 GiB | 35.8 GiB |
| Global on both sequences | 50 bases | 25.0 s | 23.5 s | 9.1 GiB | 35.8 GiB |
| Exact text | 30 bases | 64.0 s | 6.0 s | 3.1 GiB | 35.8 GiB |
| Exact text | 309 bases | 86.5 s | 7.8 s | 3.1 GiB | 35.8 GiB |
| Exact text | 1,509 bases | 72.6 s | 6.7 s | 3.1 GiB | 35.8 GiB |
| Pattern | 30 bases | 87.0 s | 8.5 s | 12.8 GiB | 35.8 GiB |
| Pattern | 309 bases | 86.5 s | 7.9 s | 12.1 GiB | 35.8 GiB |
| Pattern | 1,509 bases | 85.8 s | 7.7 s | 12.1 GiB | 35.8 GiB |
Searches an hour
The load test sends 20 searches at once: 10 proteins by Smith-Waterman on Patent proteins and 10 DNA queries by BLAST on Patent DNA, of 9 to 1,509 letters. On release 0.72.0, with 12 CPUs and 56 GiB of memory for the container and one search of the snapshot at a time, the default, all 20 finished 150.4 seconds after they were queued: about 479 searches an hour for that mix.
A steady load keeps 4 searches queued or running for 30 minutes, sending the next as one ends, with one kind of search at a time. On release 0.91.2, with 12 CPUs and 56 GiB of memory for the container, 12 threads for each search and 2 searches of the snapshot at a time:
| Method | Set | Queries | Searches an hour | Median time | 95th percentile | CPUs busy | Most memory |
|---|---|---|---|---|---|---|---|
| Automatic (Smith-Waterman) | Patent proteins | 20 proteins of 127 to 476 residues | 350 | 40.3 s | 56.6 s | 11.8 | 4.3 GiB |
| Automatic (BLAST) | Patent DNA | 20 DNA sequences of 308 to 1,911 bases | 1,184 | 11.0 s | 14.7 s | 11.3 | 7.9 GiB |
| Exact text | Patent DNA | 24 DNA fragments of 21 and 30 bases | 472 | 29.1 s | 35.4 s | 11.8 | 3.1 GiB |
| Pattern | Patent DNA | 24 DNA fragments of 21 and 30 bases | 366 | 38.1 s | 42.3 s | 11.3 | 15.3 GiB |
A search's time runs from its request to its end, its wait in the queue included. The searches an hour count those that completed within the 30 minutes, CPUs busy is the CPU time they recorded over those minutes, and the most memory is the most one of their programs held.
Sizes on disk
Sizes on disk, read November 14, 2026, 19:48 UTC:
| Files of | Sets | Files | Bytes | Size |
|---|---|---|---|---|
Snapshot 2026-11-13-5e447c75c43f | Patent DNA and Patent proteins | 44 | 100,867,162,028 | 93.9 GiB |
Snapshot 2026-11-10-37cac4ca8f21 | Swiss-Prot, TrEMBL and ESTs | 142 | 235,474,458,780 | 219.3 GiB |
| Attached BLAST database | NCBI core_nt | 835 | 318,771,562,686 | 296.9 GiB |
AWS prices
On-demand prices in US dollars an hour, for Linux in US East (N. Virginia) (us-east-1), from the AWS price list published November 14, 2026, 00:13 UTC:
| Size | vCPUs | Memory | r6i | r7i | r6a | r7a | r5 |
|---|---|---|---|---|---|---|---|
| xlarge | 4 | 32 GiB | $0.252 | $0.2646 | $0.2268 | $0.3043 | $0.252 |
| 2xlarge | 8 | 64 GiB | $0.504 | $0.5292 | $0.4536 | $0.6086 | $0.504 |
| 4xlarge | 16 | 128 GiB | $1.008 | $1.0584 | $0.9072 | $1.2172 | $1.008 |
| 8xlarge | 32 | 256 GiB | $2.016 | $2.1168 | $1.8144 | $2.4344 | $2.016 |
| 12xlarge | 48 | 384 GiB | $3.024 | $3.1752 | $2.7216 | $3.6516 | $3.024 |
| 16xlarge | 64 | 512 GiB | $4.032 | $4.2336 | $3.6288 | $4.8688 | $4.032 |
The kit's default, r6i.2xlarge, costs $0.504 an hour, $367.92 for a month of 730 hours. Its gp3 data volume costs $0.08 per GB-month: $16.00 a month for the default 200 GiB.
Audit export for a SIEM
With sign-in on, the service keeps its own audit log, and an admin can export it with every case's log as JSON lines for a SIEM, a UTC day or any time range at a time. Every event in these logs carries the SHA-256 hash of the event before it and its own hash, so a changed, missing or reordered event breaks the chain from that event on. The export carries the hashes, so you can check them.
The service's log is audit/events.jsonl on the data volume, with its last event named in audit/head.json. It holds:
- Each sign-in, refused sign-in, failed sign-in and sign-out: the time, the account as the user claim names it, its ID at your provider, and for a refused or failed one the reason in fixed words. A refused sign-in keeps at most 100 characters of the name your provider sent. A failed one names no account, since none was confirmed.
- A failed sign-in only when your provider's answer comes back to a sign-in this server started: the request to
/api/auth/callbackmust carry the state the server gave the browser at the start. Anyone can call that address, so a call without the state is not noted. During a burst, the rest of a minute's failed sign-ins are noted as onesignin.failed.suppressedevent with their count. - Each API token made or revoked, by the token's ID, with who revoked it. Never the token or its name.
- Each start: the release, the license's ID, plan, state and last day, each setting in Settings that is set, and the names of those that changed since the last start. A secret, a key, or a setting that names a secret's file shows as
set, and an address shows without its user name, query or fragment. - Each change to the invite file, when the service next reads it: the entries added and removed.
- Each export through the API, with the admin who made it and what it held.
- Each case handed to another account: the case's ID, and who handed it to whom, as the user claim names them and by their IDs at your provider. The case's own log records it too.
- On a coordinator, each worker that joins, with its name, the searches it runs at a time, its threads and the copies that match; each that leaves; each that is lost, with the IDs of the searches it ran; and each refusal, with the reason in a word. A request to the worker port without the worker key names no worker. During a burst, the rest of a minute's refusals are noted as one
worker.refused.suppressedevent with their count. - Each deletion of working files (see Working files), with the count of files and bytes and never their names or what they held:
files.deletedfor a job's files at its final state, with the job's ID, kind and state, and for what a stop left, at the next start;session.endedfor a session that ended with working files, with the session's ID, its account and why it ended:signed-out,expired, orreplacedwhen the browser signed in as another account. - Each look at the records that removed some (see How long records are kept): the days, the count of searches, exact alignments, combined searches, portfolio exports and batches removed, and their bytes.
- Each case an admin deletes: the admin, the case's ID, its owner's ID at your provider, the number of events in the case's log and the SHA-256 of the last one, the count of searches deleted with it and their bytes. The case's own log goes with the case.
It holds no IP addresses, queries, sequences or results. Searches, page views and the container's commands (such as load or sample add) are not in it; the usage page counts searches. The service only adds to the log, with one exception: a last line that a stop cut off before its end, which never became an event, is taken off by the next event. Nothing rotates or deletes the log, and it grows by about 300 bytes a sign-in. audit/head.json is a cache of the log's last line: after a stop between an event's line and head.json, the check below says the log's head does not name its last event until the next event, which follows the line. Each case's log, with its head in the case's case.json, and each account's join log work the same way. With sign-in off there is no log.
To export a day, with an admin's API token (see API tokens for programs):
curl -H "Authorization: Bearer <token>" -o sequences-audit-2026-11-12.jsonl \
"https://sequences.example.org/api/admin/audit.jsonl?day=2026-11-12"
dayis a UTC date. For another range, givefromandtoinstead, as times with a time zone, such asfrom=2026-11-12T00:00:00Z&to=2026-11-12T12:00:00Z.fromis in the range andtois not.- Only the people in
BIOSEQUENCE_ADMINScan export. For anyone else, and with sign-in off, the address does not exist. - The container's
audit-exportcommand writes the same lines from the data volume, which it only reads, so it can run beside the service, from a scheduled job on the server. The service's log does not note these exports. The command exits with 1 after writing when a log's chain is broken:
docker run --rm -v sequences-data:/data:ro conductscience/sequences:<version> audit-export --day 2026-11-12 \
> sequences-audit-2026-11-12.jsonl
Each line is one JSON object, and its record says what it is:
event: one event.lognames its log:service,case/<case ID>, orjoins/<folder>for an account's applicant joins. For the service's log,eventis the event as stored. For a case or joins log it keeps the event's number, time, user, type, IDs, counts and hashes, andwithheldnames the fields left in the log. Case names, claim text, notes, reasons, the words of summaries and other text stay out, since claims pasted into a case can be from an application that is not yet published.chain, after each log's events:firstandlast, the numbers of the events given;prev, the hash of the event beforefirst, or null at the log's start;head, the last event the log's record names; andcheck, the service's check of the whole log as it exported it, withokfalse and the first broken event when it fails.export, last: the range, when the file was made, and the numbers of events, logs and broken logs, with the logs it could not read inskipped.
A log's events in a file run from its first event in the range to its last, so they always follow one another in its chain, and the next day's file goes on from there.
To check exports, save the script below as check_audit.py. It needs Python 3 and nothing else. Give it the exports oldest first. It checks that each log's events follow one another and that each event's prev is the hash of the one before, across files too, and it recomputes the hash of each event given whole. With --data and the data volume, it also reads each log whole, recomputes every hash, checks each log's head, and checks each exported event against the log. The image has Python 3, so the second check can run there with the volume read-only:
python3 check_audit.py sequences-audit-2026-11-12.jsonl sequences-audit-2026-11-13.jsonl
docker run --rm -v sequences-data:/data:ro -v "$PWD":/check:ro --entrypoint python3 \
conductscience/sequences:<version> /check/check_audit.py /check/sequences-audit-2026-11-12.jsonl --data /data
It prints each problem it finds and exits with 1, or ends with "the chains hold" and exits with 0. The script:
# check_audit.py checks Sequences audit exports with Python 3 and its standard library alone.
# python3 check_audit.py EXPORT [EXPORT ...] [--data FOLDER]
# Give exports oldest first. In each log it checks that the events follow one another, that each event's prev is the
# hash of the event before it, across exports too, and that each event exported whole still has its own hash. With
# --data and the data folder, it reads each log whole, recomputes every hash, checks the log's head, and checks that
# each exported event is the log's.
import argparse, decimal, hashlib, json, os, re, sys
SURROGATE = re.compile('[\ud800-\udfff]')
LOG = re.compile(r'(service|case/[0-9a-f-]{36}|joins/([0-9a-f]{64}|local))$')
NO_HEAD = {'seq': 0, 'hash': None}
def number(x):
# A float as JavaScript writes it, the shortest digits that read back the same.
if x == 0:
return '0'
t = decimal.Decimal(repr(abs(x))).as_tuple()
digits = ''.join(map(str, t.digits))
n = len(digits) + t.exponent
digits = digits.rstrip('0')
k, sign = len(digits), '-' if x < 0 else ''
if k <= n <= 21:
return sign + digits + '0' * (n - k)
if 0 < n <= 21:
return sign + digits[:n] + '.' + digits[n:]
if -6 < n <= 0:
return sign + '0.' + '0' * -n + digits
e = n - 1
return sign + digits[0] + ('.' + digits[1:] if k > 1 else '') + 'e' + ('+' if e >= 0 else '-') + str(abs(e))
def text(s):
return SURROGATE.sub(lambda m: '\\u%04x' % ord(m.group()), json.dumps(s, ensure_ascii=False))
def canonical(v):
# JSON as the service hashes it: as JavaScript writes it, with the keys of every object sorted.
if v is None or isinstance(v, bool):
return json.dumps(v)
if isinstance(v, int):
return str(v)
if isinstance(v, float):
return number(v)
if isinstance(v, str):
return text(v)
if isinstance(v, list):
return '[' + ','.join(map(canonical, v)) + ']'
keys = sorted(v, key=lambda key: key.encode('utf-16-be'))
return '{' + ','.join(text(key) + ':' + canonical(v[key]) for key in keys) + '}'
def sealed(event):
rest = {key: value for key, value in event.items() if key != 'hash'}
return hashlib.sha256(canonical(rest).encode('utf-8')).hexdigest() == event.get('hash')
def lines_of(path):
lines = []
with open(path, encoding='utf-8', newline='') as f:
for n, line in enumerate(f.read().split('\n'), 1):
if line.strip():
try:
value = json.loads(line)
except ValueError:
value = None
lines.append(value if isinstance(value, dict) else {'invalidLine': n})
return lines
def check_exports(exports, problems):
seen = {}
for n, lines in enumerate(exports, 1):
def say(log, seq, reason):
problems.append('export %d%s%s: %s' % (n, ', ' + log if log else '', ', event %s' % seq if seq else '', reason))
if any('invalidLine' in line for line in lines):
say(None, None, 'a line is not a JSON object')
tail = lines[-1] if lines else {}
if tail.get('record') != 'export' or tail.get('format') != 'sequences-audit':
say(None, None, 'it has no export line at its end, so it may be cut short')
logs = {}
for line in lines:
if line.get('record') in ('event', 'chain'):
logs.setdefault(line.get('log'), []).append(line)
for log, group in logs.items():
events = [line for line in group if line['record'] == 'event']
chain = next((line for line in group if line['record'] == 'chain'), None)
if chain is None:
say(log, None, 'it has no chain line')
continue
known = seen.setdefault(log, {})
first, prev, last = chain.get('first') or 0, chain.get('prev'), max(known, default=0)
if last and first > last + 1:
say(log, last + 1, 'events %d to %d are in none of the exports given' % (last + 1, first - 1))
if first - 1 in known and known[first - 1] != prev:
say(log, first, 'it does not follow the event an earlier export gave before it')
if len(events) != (chain.get('last') or 0) - first + 1:
say(log, None, 'its chain line and its events disagree on how many there are')
for seq, line in enumerate(events, first):
event = line.get('event')
if not isinstance(event, dict) or event.get('seq') != seq:
say(log, seq, 'this line is not the event with this number')
break
if event.get('prev') != prev:
say(log, seq, 'it does not follow the event before it')
if not line.get('withheld') and not sealed(event):
say(log, seq, 'it changed after it was written')
if known.get(seq, event.get('hash')) != event.get('hash'):
say(log, seq, 'an earlier export gave it another hash')
known[seq] = prev = event.get('hash')
check = chain.get('check') or {}
if check.get('ok') is False:
say(log, None, 'the server found the log broken when it exported it: %s' % (check.get('broken') or {}).get('reason'))
def read_log(data, log):
if not LOG.match(log or ''):
return None
kind, _, name = log.partition('/')
folder = os.path.join(data, {'service': 'audit', 'case': 'cases', 'joins': 'applicants'}[kind], name)
if not os.path.exists(os.path.join(folder, 'events.jsonl')):
return None
if kind == 'case':
with open(os.path.join(folder, 'case.json'), encoding='utf-8') as f:
head = json.load(f).get('head') or NO_HEAD
elif os.path.exists(os.path.join(folder, 'head.json')):
with open(os.path.join(folder, 'head.json'), encoding='utf-8') as f:
head = json.load(f)
else:
head = NO_HEAD
return lines_of(os.path.join(folder, 'events.jsonl')), head
def broken(events, head):
prev = None
for seq, event in enumerate(events, 1):
if event.get('seq') != seq:
return seq, 'this line is not the event with this number'
if event.get('prev') != prev:
return seq, 'it does not follow the event before it'
if not sealed(event):
return seq, 'it changed after it was written'
prev = event['hash']
if (head.get('seq'), head.get('hash')) != (len(events), prev):
return min(head.get('seq') or 0, len(events)) + 1, 'the log\'s head does not name its last event'
return None
def check_data(exports, data, problems):
logs = {}
for lines in exports:
for line in lines:
if line.get('record') == 'event':
logs.setdefault(line.get('log'), []).append(line)
for log, group in logs.items():
found = read_log(data, log)
if found is None:
problems.append('%s: the data folder has no such log' % log)
continue
events, head = found
fault = broken(events, head)
if fault:
problems.append('%s, event %d: %s' % (log, fault[0], fault[1]))
for line in group:
event = line.get('event') or {}
seq, withheld = event.get('seq'), line.get('withheld') or []
stored = events[seq - 1] if isinstance(seq, int) and 0 < seq <= len(events) else {}
if not event or stored.get('hash') != event.get('hash'):
problems.append('%s, event %s: the log does not hold it' % (log, seq))
continue
for key in set(stored) | set(event) | set(withheld):
if (key in event) == (key in withheld) or key not in stored or (key in event and canonical(event[key]) != canonical(stored[key])):
problems.append('%s, event %d: its %s does not match the log' % (log, seq, key))
def main():
parser = argparse.ArgumentParser(description='Check Sequences audit exports.')
parser.add_argument('exports', nargs='+', help='the export files, oldest first')
parser.add_argument('--data', help='the data folder, to check the logs themselves')
args = parser.parse_args()
exports = [lines_of(path) for path in args.exports]
problems = []
check_exports(exports, problems)
if args.data:
check_data(exports, args.data, problems)
for problem in problems:
print(problem)
events = sum(line.get('record') == 'event' for lines in exports for line in lines)
print('%d exports, %d events: %s' % (len(exports), events, '%d problems' % len(problems) if problems else 'the chains hold' + (', and every event matches the data folder' if args.data else '')))
return 1 if problems else 0
if __name__ == '__main__':
sys.exit(main())
Sample results
A sample result is a finished search saved as a read-only page, so the people you invite can see what a search finds before they sign in. While there is one, the sign-in page and the menu at the top of every page link to the samples at /samples. Anyone who can reach the service can read a sample without signing in, so make samples only from searches of public sequences. On your server the samples carry noindex, so search engines do not list them.
Make one from a finished search of a patent set, named by its ID (the job in the search page's address) or its first 8 or more characters:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> sample add <search ID> --name <name> \
--title "<what it shows>" --source "<the public record>" --source-url <https link to the record>
The name, up to 40 lowercase letters, digits and hyphens, gives the page its address, /samples/<name>. The command copies the search's hits, alignments and records into samples/ on the data volume, so a sample stays when its snapshot is replaced. It copies nothing about who ran the search and no marks. It refuses a search that belongs to a case, a batch or a combined search, an antibody regions search, and text that looks like an email address. sample list lists the samples, and sample remove <name> removes one. The service shows each change without a restart.
Watch emails
People can watch a finished search of the snapshot, or an applicant. When you load a delta, the service runs each watch on a search again within the hour and lists the matches it has not found before under Your watches on the search page. A watch on an applicant runs no search: within the hour it lists the applicant's publications that the delta adds.
With sign-in on, the service can also email people when a watch finds something new. It sends email only through Resend, an email delivery service, so you need a Resend account, a sender address on a domain you have verified there, and both BIOSEQUENCE_MAIL_FROM and BIOSEQUENCE_RESEND_KEY. With one setting alone the service does not start, and the log names the missing one. With neither, no email goes out.
BIOSEQUENCE_MAIL_DETAILS sets what each email holds, for every watch on the server:
- Unset or
off, the default: only how many new matches or publications one of the person's watches found, such as "One of your watches has 3 new matches. Sign in to see them.", with a link to Your watches on the search page. The email names no watch, query or match. on: the watch's name, its set or applicant, the date of the newest delta it ran on, and up to 50 of its new matches or publications with their titles. A new publication links to its patent, and a watch on a search links to the run's results, which open after sign-in.
Any other value stops the service at start, and the log names the setting. Each start's log says whether watch emails go out, and with or without details.
- Each email goes to the address in the person's email claim, so emails need
BIOSEQUENCE_OIDC_USER_CLAIM=email, the default. With another user claim, such aspreferred_username, watches show what they find on the page only. - Each email has a link that pauses the watch without signing in.
- With sign-in off there is no address to send to, so the service sends no email.
Claim suggestions from a model
A case's Review step can ask a language model for claim features the rules missed: SEQ ID references, identity thresholds, fragment lengths, residue ranges and written-out sequences. Suggestions are off until you name a model. Each one names the model and the exact characters it came from, and counts only once someone accepts it. The button that asks reads "(Pilot)".
Run the model on a server you control, behind a server that takes chat completions calls, such as vLLM, llama.cpp's server or Ollama, and set:
BIOSEQUENCE_CLAIMS_MODEL_URL=http://127.0.0.1:8000/v1/chat/completions
BIOSEQUENCE_CLAIMS_MODEL_API=openai-chat
BIOSEQUENCE_CLAIMS_MODEL=your-model-name
The service uses the address as written, so give the whole path. With sign-in on, 127.0.0.1 is the container itself: start the model's container with --network container:sequences, which shares that address, or name the model's host in BIOSEQUENCE_CLAIMS_MODEL_HOSTS and serve it over HTTPS. At start the log names the model, where it runs, the format and how long the service waits for it, and never the key.
The model receives the case's claims, the features the rules found in each, and, when the claims came from a publication's XML, up to 40 paragraphs of its description, 60,000 characters at most, that cite a SEQ ID NO the claims name or define a term the claims use. It is asked for JSON. A suggestion stays only when its quote is in its claim. Its details stay only when each number in them is written in that quote, and a quote from the description stays only when it is in the paragraph it names. When one fails, the suggestion stays without it, and the reason is recorded.
A model that runs on processors alone can take minutes for one case, so with openai-chat the service waits up to 900 seconds unless BIOSEQUENCE_CLAIMS_MODEL_SECONDS says otherwise. The call runs in the background: the page shows how long it has been asking, and people keep working on the case. The service makes one call at a time. While one runs, a request from any other case is refused with a message to ask again in a few minutes.
Each call is recorded in its case, whether it brings suggestions or fails: the request as sent and its SHA-256, the model's answer up to 1,000,000 characters, how long the call took, and each suggestion kept or dropped with the reason. The case's log holds the same record. A key, if you set one, goes only in the Authorization header and is never recorded.
Summary drafts
The same settings let a person ask the model for a draft of a case's text (Pilot): the case summary, the results summary, a claimed SEQ ID NO's summary or the sentences on a kept publication. Drafts are off until you name a model, as suggestions are. Fixed templates still write each text, and the template's words stay until someone accepts or edits a draft. The model receives only that text's facts, the ones its template reads, each with its source and at most 20,000 characters in all, and is asked for at most 150 words in at most 2,048 answer tokens. A draft is dropped when a span it quotes is not in a source it was given, or a number in it is not in the facts (a percent to one decimal place). The page then says which check failed and never shows the draft's words.
Drafts and claim suggestions share the model: one call at a time for the whole server, with the same time limit. Nothing drafts by itself; each draft is asked for by a person. The case's log records each call with the model, the prompt's ID and SHA-256, the tokens and the seconds, and each decision, and a report names the model for a summary it drafted.
Paper citations and marks
A paper folder gives each paper a hit cites its PubMed citation and the day it first came out, online or in print, so Published before compares days rather than years. It also marks papers that were retracted, drew an expression of concern or were reinstated, from PubMed and the Retraction Watch Database. With PubTator 3 in it, a case can draft its paper sentence: the genes and diseases a paper names. The papers command makes the folder from files you download, with no network connection, and the service only reads it.
Download these into /srv/papers on the server. PubMed's two folders held about 75 GB in November 2026, and NLM's terms are in the README.txt of each.
| Path | What to download |
|---|---|
/srv/papers/baseline/ | PubMed's baseline files, pubmed*.xml.gz, from https://ftp.ncbi.nlm.nih.gov/pubmed/baseline/ |
/srv/papers/updatefiles/ | PubMed's update files, pubmed*.xml.gz, from https://ftp.ncbi.nlm.nih.gov/pubmed/updatefiles/ |
/srv/papers/deleted.pmids.gz | NLM's list of deleted PubMed IDs, from https://ftp.ncbi.nlm.nih.gov/pubmed/deleted.pmids.gz |
/srv/papers/retraction_watch.csv | The file of that name from Crossref's Retraction Watch data repository, https://gitlab.com/crossref/retraction-watch-data. Note the commit you took it from and that commit's date |
/srv/papers/bioconcepts2pubtator3.gz | If you like, about 6 GB: PubTator 3's genes and diseases, from https://ftp.ncbi.nlm.nih.gov/pub/lu/PubTator3/. Note the date the folder lists for the file |
Then make the folder. --out is the parent folder: the command makes a folder named by its ID inside it, and prints the ID at the end. Give --snapshot once for each snapshot you search. A snapshot's ID is the name of its folder under corpora/. Give the Retraction Watch commit and its date, and PubTator's date, as you noted them. If you skipped PubTator 3, leave out --pubtator and --pubtator-date. Without the Retraction Watch file, leave out its three options, and the marks come from PubMed alone. The command uses one thread for each CPU, up to 12, unless you give --threads. Allow the container 12 GB of memory:
docker run --rm --memory 12g -e NODE_OPTIONS=--max-old-space-size=6144 \
-v sequences-data:/data -v /srv/papers:/in:ro conductscience/sequences:<version> \
papers --out /data/papers --snapshot /data/corpora/<snapshotId> \
--pubmed /in/baseline --pubmed /in/updatefiles --deleted /in/deleted.pmids.gz \
--retraction-watch /in/retraction_watch.csv --retraction-watch-commit <commit> --retraction-watch-date <YYYY-MM-DD> \
--pubtator /in/bioconcepts2pubtator3.gz --pubtator-date <YYYY-MM-DD>
Set BIOSEQUENCE_PAPERS=/data/papers/<id>, with the ID the command printed, then remove the container and start it again. The log then says Papers come from paper folder <id>: ..., and Sources and terms credits PubMed, and the Retraction Watch Database and PubTator 3 when the folder holds them. If the folder is not a finished paper folder, the service does not start, and the log says why.
A search keeps the paper entries it had when it ran, so a search from before the folder shows no marks until it runs again. For newer PubMed files or a new snapshot, make a new folder and point the setting at it. Without the setting, each paper keeps the year its snapshot gives it.
Dates from Crossref
A paper that PubMed does not hold can still take its date from Crossref when it has a DOI. These are the papers cited by DOI alone, and those whose PubMed ID NLM deleted or PubMed's files never held. The crossref command asks Crossref's REST API, https://api.crossref.org, for the dates of each such DOI, one at a time and at most 3 a second. It keeps the answers in the folder --answers names, so a later run asks only about DOIs it has no answer for. Give --snapshot as for papers, and --papers with the paper folder you have, if any, so that it also asks about the PubMed IDs that folder lists as deleted or not held:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
crossref --answers /data/crossref --snapshot /data/corpora/<snapshotId> --papers /data/papers/<id>
Crossref asks automated clients for a contact address. The command names itself Sequences-snapshot/<version> and sends no address unless you set BIOSEQUENCE_CROSSREF_MAILTO, for example with -e BIOSEQUENCE_CROSSREF_MAILTO=<address> in the command above. A first run takes about a second for every three DOIs. Then make the paper folder as above, with --crossref /data/crossref added to the papers command; that command still makes no network connection. For each DOI the folder takes the earliest of the print, online and issued dates Crossref gives, to the day or the month it names. When it names only a year, the paper keeps its year. A paper dated this way reads, for example, "Published 2021-12-01 (Crossref)." under the hit, its date in the hits CSV's paper_dates column ends in "(Crossref)", and Sources and terms credits Crossref.
Species of each hit
Each hit lists the organisms its records name. A taxa folder also gives each record its NCBI taxon and a group, such as Human, Mouse or Artificial or synthetic, so the page and the API can filter a search's hits by species or group. The taxa-scan and taxa commands make the folder from NCBI's taxonomy and the source files of your snapshots, with no network connection, and the service only reads it.
Download NCBI's taxonomy, taxdump.tar.gz, about 80 MB, and its checksum file, taxdump.tar.gz.md5, from https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/ into /srv/taxa on the server, readable by user ID 10001. Check the file before you use it:
cd /srv/taxa && md5sum -c taxdump.tar.gz.md5
Then scan the source files of each snapshot you built, before you remove them, as in Build from the public files. A scan reads the NCBI taxon each record states. Give --inputs for the snapshot's folder, then once more for each of its deltas' folders, in order. With --source all, the command scans each source those folders list into a folder of its own inside --out, a new folder:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
taxa-scan --inputs /data/inputs/release-274.0 --source all --out /data/taxa-scans/release-274.0
A snapshot you loaded as a copy has no source files on the server. Skip the scans. Each of its records keeps the taxon and group the snapshot was built with, or, if it was built without a taxa folder, is placed by its organism name alone.
Then make the folder. --out is the parent folder: the command makes a folder named by its ID inside it, and prints the ID at the end. A taxa folder holds one scan of each source. Give --scans for each folder of scans you made, or leave it out if you made none:
docker run --rm -v sequences-data:/data -v /srv/taxa:/in:ro conductscience/sequences:<version> \
taxa --taxdump /in/taxdump.tar.gz --out /data/taxa --scans /data/taxa-scans/release-274.0
Set BIOSEQUENCE_TAXA=/data/taxa/<id>, with the ID the command printed, then remove the container and start it again. The log then says Species groups come from taxa folder <id>, built <time>., and Sources and terms credits NCBI Taxonomy. If the folder is not a finished taxa folder, the service does not start, and the log says why.
To keep each record's taxon and group in a snapshot or delta you build, give build or delta the folder with --taxa /data/taxa/<id>. For a new snapshot, scan its source files and make a new folder first. A copy of the snapshot loaded on another server keeps them too.
A search keeps the species it had when it ran, so a search from before the folder cannot be filtered by species until it runs again. Keep /data/taxa-scans: with a newer taxdump.tar.gz, the taxa command alone makes a new folder. Without the setting, each hit still lists the organisms its records name, with no taxon or group.
ROR suggestions
The Applicants tab can suggest the record in the Research Organization Registry (ROR) of an applicant whose name matches an organization's name there. A suggestion is only a link: it never joins or renames applicants. It needs a name index on your server, made once for each ROR release.
Download ROR's data file, a zip of about 39 MB under CC0, from https://doi.org/10.5281/zenodo.6347574 into a folder on the server, such as /srv/ror. Then make the index in the data volume. The command reads the zip and makes no network connection. Here --out names the index folder itself, not its parent. Name it after the JSON file in the zip, which unzip -l lists: v2.15-2026-10-20-ror-data.json gives v2.15-2026-10-20.
docker run --rm -v sequences-data:/data -v /srv/ror:/in:ro conductscience/sequences:<version> \
ror --zip /in/<ROR zip> --out /data/ror/v2.15-2026-10-20
Set BIOSEQUENCE_ROR=/data/ror/v2.15-2026-10-20, then remove the container and start it again. The log then says Applicant suggestions from ROR v2.15, data of 2026-10-20., and Sources and terms credits ROR. If the folder is not a finished index, the service does not start, and the log says why. For a newer ROR release, make a new folder and point the setting at it. Without the setting, the Applicants tab works and suggests nothing.
Patent context
What the patent says, in a US publication's pane, quotes its abstract, its claims that name a matching SEQ ID NO, up to three description paragraphs for each and its sequence listing's notes. See What the patent says. The quotes come from a patent context folder that the context command makes from the US patent office's weekly full-text files. The command reads the files you downloaded and makes no network connection. The service only reads the folder, so a search takes no longer and the page asks no other site.
The command reads the weekly ZIP files of applications (APPXML) and grants (PTGRXML, and PTGRSGM for the grants of 2001 in SGML), from the US patent office's Open Data Portal, which asks for an API key of your own. Keep them in one folder, such as /srv/patent-text, in the paths of the bulk data: a folder for each kind and a folder for each year in it, such as APPXML/2026/ipa261105.zip, with PTGRSGM/ holding its files directly. Where a week has a revised file, such as ipa261105_r1.zip, the command takes the newest. In November 2026 the files since 2001 held about 282 GB.
Then make the folder in three steps. extract reads each weekly file once into a part, one file to a thread, and passes over the files it has done, so a run that stops starts again where it stopped. assemble writes the folder for the US publications of one snapshot and of the deltas you name with --delta. check reads the folder as the service will and checks its SHA-256:
docker run --rm -v sequences-data:/data -v /srv/patent-text:/in:ro conductscience/sequences:<version> \
context extract --archive /in --parts /data/context-parts --workers 8
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
context assemble --parts /data/context-parts --snapshot /data/corpora/<snapshotId> --out /data/context/<snapshotId>-1
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
context check --folder /data/context/<snapshotId>-1
One weekly application file of 183 MB took 8 seconds on one thread, at most 1.3 GB of memory, and gave a part of 1.1 MB. At that speed all the files since 2001 take about 3 hours on one thread, and less with more workers as far as the disk keeps up; their parts come to about 2 GB. Allow the container 2 GB of memory for each worker. If a worker runs out of memory on a large file, the summary names the file: run extract again with fewer workers and a larger heap each, which -e NODE_OPTIONS=--max-old-space-size=<MB> sets, and it reads only the files that have no part.
Set BIOSEQUENCE_CONTEXT=/data/context/<snapshotId>-1, then remove the container and start it again. The log then says Patent context from the folder of <date>: <count> US publications of snapshot <snapshotId>. If the folder does not pass its checks, the service does not start, and the log says why. When a delta adds a week, run extract again for the new files, assemble into a new folder with a --delta for each delta, and point the setting at it. Without the setting, a pane shows no What the patent says.
Other offices
For publications of other offices, such as WO, EP, JP and KR, the part can show the title and the English abstract that EPO's Open Patent Services (OPS) gives, with a link to Espacenet for the claims and the description. OPS needs an account of your own at EPO, under its fair use terms and weekly limit. This is off by default: context ops asks OPS only when all three of these settings are given, and otherwise sends nothing.
| Setting | What it holds |
|---|---|
BIOSEQUENCE_OPS | on lets context ops ask OPS. Unset or any other value, the command stops and asks nothing |
BIOSEQUENCE_OPS_KEY_FILE | A file that holds your OPS consumer key, such as /config/ops-key |
BIOSEQUENCE_OPS_SECRET_FILE | A file that holds your OPS consumer secret, such as /config/ops-secret |
Keep the two files on the server, readable by the container's user only, as you keep the sign-in secret. The command sends OPS the numbers of the snapshot's publications of other offices, one request at a time and slower when OPS asks, and nothing about searches or the people who run them. It keeps what it has asked, so a run that stops or reaches the weekly limit starts again where it stopped. Run it with the parts folder of extract:
docker run --rm -e BIOSEQUENCE_OPS=on -e BIOSEQUENCE_OPS_KEY_FILE=/config/ops-key \
-e BIOSEQUENCE_OPS_SECRET_FILE=/config/ops-secret -v /etc/sequences:/config:ro -v sequences-data:/data \
conductscience/sequences:<version> context ops --snapshot /data/corpora/<snapshotId> --parts /data/context-parts
Then run assemble into a new folder and point BIOSEQUENCE_CONTEXT at it, as above. The log's line then adds and <count> of other offices.
Antibody names
Antibody CDR search can name the therapeutic antibodies each match shares sequence with: those whose whole heavy or light variable domain the stored sequence holds, or whose three CDRs of one chain it holds letter for letter, in order, under the search's numbering. The names come from ChEMBL, of EMBL's European Bioinformatics Institute (EMBL-EBI), and need a name table on your server, made once for each ChEMBL release. With several servers, only the coordinator needs it.
The name table is an adaptation of ChEMBL's data, so it is released under ChEMBL's license, Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0), which covers the table and nothing else in Sequences. Its folder holds the license's legal code as LICENSE.txt and ChEMBL's credit as ATTRIBUTION.txt. Keep both with the table wherever you copy it, and keep the table in a folder of its own. The credit ChEMBL asks for is its address and release, as in ChEMBL data is from http://www.ebi.ac.uk/chembl - the version of ChEMBL is chembl_37., and a citation of its paper: Zdrazil B, et al. The ChEMBL Database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods. Nucleic Acids Res. 2024;52(D1):D1180-D1192, https://doi.org/10.1093/nar/gkad1004.
Make the table in two steps. The first downloads one release's biotherapeutic sequences (chembl_37_bio.fa.gz, about 220 KB) with its LICENSE, README and checksums.txt from https://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/releases/, checks the file's SHA-256, the license and the credit line, then asks ChEMBL's web services (https://www.ebi.ac.uk/chembl/api/data/) for each molecule's names, one request a second, about 30 in all. The web services answer from their current release only, so fetch that one, which https://www.ebi.ac.uk/chembl/api/data/status.json names. The second step numbers every chain as the CDR search does, keeps each variable domain with its CDRs under Kabat, Chothia and IMGT, and makes no network connection. Each --out names a new folder itself, not its parent.
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
antibody-names fetch --release 37 --out /data/chembl/chembl_37
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
antibody-names build --from /data/chembl/chembl_37 --out /data/antibody-names/chembl_37
Before you set it, check the table with the image you run. antibody-names check opens it as the service does and names each chain of a FASTA file under Kabat, Chothia and IMGT. With --expect, it exits with 1 unless every chain names that antibody under all three. The image holds trastuzumab's two variable domains from the guide's demo:
docker run --rm -v sequences-data:/data conductscience/sequences:<version> \
antibody-names check --table /data/antibody-names/chembl_37 \
--chains /app/services/biosequence/demo/trastuzumab-chains.fasta --expect Trastuzumab
Set BIOSEQUENCE_ANTIBODY_NAMES=/data/antibody-names/chembl_37, then remove the container and start it again. The log then says Antibody names from ChEMBL chembl_37 (CC BY-SA 3.0): ... with the number of named antibodies and variable domains, each named match credits ChEMBL and the license, and Sources and terms credits ChEMBL. If the folder is not a finished table, or its LICENSE.txt or ATTRIBUTION.txt is missing or changed, the service does not start, and the log says why. For a newer ChEMBL release, make new folders and point the setting at the new table. Without the setting, CDR search works and names no antibodies.
Clinical context
Clinical context is an add-on: it runs only when BIOSEQUENCE_ADDONS names it. Under a protein hit whose UniProt entry is a drug target, it shows the drugs that act on the target with their mechanisms, the diseases each is studied for with the highest clinical stage, and the trials, with their phase and whether they posted results. It shows on hits of the Swiss-Prot and TrEMBL sets. Patent, EST and attached database hits show none. Its data come from Open Targets (with ChEMBL), ClinicalTrials.gov and AACT's copy of trial results, in one folder that ConductScience builds for each release of them. Without the add-on, the service reads no such folder, and the page and the API show none of it.
The folder comes as a copy with its SHA-256 list, as a snapshot does, and holds tens of megabytes. To get it, write to support@conductscience.com. Put both in the import folder and make them readable, as in Load a copy, then load the copy into the data volume. load checks every file against the list, opens the folder as the service will and makes it read-only at /data/clinical/<release>, under the release name its folder carries. It makes no network connection.
docker run --rm \
-v sequences-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/<release> --sums /import/<release>.SHA256SUMS
Set BIOSEQUENCE_ADDONS=clinical and BIOSEQUENCE_CLINICAL=/data/clinical/<release>, then remove the container and start it again. The log then says Clinical context from <release>: ... with the releases of its data, and Sources and terms credits Open Targets, ChEMBL, ClinicalTrials.gov and AACT. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer release of the data, load its folder and point BIOSEQUENCE_CLINICAL at it. To stop showing clinical context, remove both settings, then remove the container and start it again.
Funding trail
The funding trail is an add-on: it runs only when BIOSEQUENCE_ADDONS lists funding. It lists the grants PubMed gives for each paper under a hit. Grants that match an NIH project show as NIH awards, with the project number linked to NIH RePORTER, the title, the institutions and fiscal years, and the investigators. The other grants follow as PubMed lists them. It shows no award amounts. ConductScience builds its data, PubMed's grant lists and NIH RePORTER's awards, into one folder, and each new build is a new release of the data. Without the add-on, the service reads no such folder, and the page and the API show none of it.
A folder can also hold the statements on federally sponsored research in US published applications with a sequence listing, word for word. Where it does, up to three of them follow a hit's papers, under a line that names them, from the earliest applications that hold the hit's sequence, each in a part named by its application. The hit row in the API carries them as statements, and, when more of its applications give one, the number of the rest as statementsMore. A statement is not matched to an NIH award or to a paper's grants. A folder without statements works as before. Each week, a new release of the folder adds that week's applications to those of the weeks before. Upgrade the image first, as in Upgrades: an older image loads a folder with statements but shows none of them, and its log line leaves out their count.
The folder comes as a copy, as a snapshot does: the folder itself and its <release>.SHA256SUMS file. It holds tens of megabytes. To get it, write to support@conductscience.com. Put the folder and its SHA-256 list in the import folder and make them readable, as in Load a copy, then load the copy into the data volume. load checks every file against the list, opens the folder as the service will, and stores it read-only at /data/funding/<release>, where <release> is the release name the folder was built under. It makes no network connection.
docker run --rm \
-v sequences-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/<release> --sums /import/<release>.SHA256SUMS
Add funding to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_FUNDING=/data/funding/<release>, then remove the container and start it again. A server that also runs clinical context needs BIOSEQUENCE_ADDONS=clinical,funding. The log then says Funding trail from <release>: ... with the dates of its data, and with statements it ends with how many applications they cover and from which weeks. Sources and terms credits PubMed and NIH RePORTER, and the US published applications when the folder holds their statements. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer release of the data, load its folder, point BIOSEQUENCE_FUNDING at it, then remove the container and start it again. The old folder can stay, and nothing reads it. To stop showing the funding trail, take funding out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_FUNDING, then remove the container and start it again.
Known variants
Known variants is an add-on: it runs only when BIOSEQUENCE_ADDONS lists variants. On a protein hit identical to a human Swiss-Prot entry, it shows the variants ClinVar records on the protein. For a protein query, it also shows where the query differs from the protein and whether ClinVar records each change. Its data come from ClinVar, RefSeq's human proteins and UniProtKB/Swiss-Prot, which ConductScience builds into one folder. Each new build is a new release of the folder. Without the add-on, the service reads no such folder, and neither the page nor the API shows known variants.
Like a snapshot, the folder comes with its <release>.SHA256SUMS file. It holds about 670 MB. To get it, write to support@conductscience.com. Put the folder and its <release>.SHA256SUMS file in the import folder and make them readable, as in Load a copy. Then load them into the data volume. The load command checks every file against the <release>.SHA256SUMS file, opens the folder the way the service will, and stores it read-only at /data/variants/<release>, where <release> is the folder's release name. It makes no network connection.
docker run --rm \
-v sequences-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/<release> --sums /import/<release>.SHA256SUMS
Add variants to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_VARIANTS=/data/variants/<release>, then remove the container and start it again. If the server also runs the clinical context and funding trail add-ons, set BIOSEQUENCE_ADDONS=clinical,funding,variants. The log then says Known variants from <release>: ClinVar of <date>, RefSeq human proteins of <date>, UniProtKB/Swiss-Prot <version>. Sources and terms credits ClinVar, RefSeq and UniProt. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer release, load its folder, point BIOSEQUENCE_VARIANTS at it, then remove the container and start it again. Searches made with the older release keep the known changes they found. To stop showing known variants, take variants out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_VARIANTS, then remove the container and start it again.
Drug names
Drug names is an add-on: it runs only when BIOSEQUENCE_ADDONS lists drugs. A hit whose sequence is the same as one of a drug's chains names the drug, with its highest stage and a link to its ChEMBL compound page or its FDA UNII record. The same means the whole chain, letter for letter; an antibody's heavy or light chain variable domain, as ANARCI numbers it, held whole in the hit; or an oligonucleotide's bases, with U read as T. A near match names no drug. The drug chains come from ChEMBL and from the public data of GSRS, the substance registry of the FDA and NCATS, in one folder that ConductScience builds for each release of them. Without the add-on, the service reads no such folder, and new searches name no drugs.
A search names drugs as it runs, and its job keeps them, so it shows the same names after the folder is replaced or the add-on is turned off. A search from before the add-on ran names none until it is run again. Hits of an attached database name none.
The folder holds ChEMBL's data, so it is shared under the Creative Commons Attribution-ShareAlike 3.0 Unported license (CC BY-SA 3.0). Its LICENSE.txt is the license, and its ATTRIBUTION.txt credits each source. Keep both with the folder. An export that names drugs credits their sources and license in its drug_sources column.
The folder comes as a copy with its SHA-256 list, as a snapshot does. To get it, write to support@conductscience.com. Put both in the import folder and make them readable, as in Load a copy, then load the copy into the data volume. load checks every file against the list, opens the folder as the service will and makes it read-only at /data/drugs/<release>, under the release name its folder carries. It makes no network connection.
docker run --rm \
-v sequences-data:/data \
-v /srv/sequences/import:/import:ro \
conductscience/sequences:<version> \
load /import/<release> --sums /import/<release>.SHA256SUMS
Add drugs to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_DRUGS=/data/drugs/<release>, then remove the container and start it again. On several servers, give each worker the same two settings and load the folder on it too, since workers run the searches. The log then says Drug names from <release>: ... with the releases of its data, and Sources and terms credits ChEMBL and GSRS. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer release of the data, load its folder, point BIOSEQUENCE_DRUGS at it, then remove the container and start it again. To stop naming drugs, take drugs out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_DRUGS, then remove the container and start it again.
Drug sets
The drug chains can also be searched as sets of their own: Drug proteins, the protein chains of drugs, and Drug nucleic acids, the strands of oligonucleotide drugs, written 5' to 3'. They come as a snapshot of their own, so no record from ChEMBL enters a patent snapshot. Each record is one chain, named by its drug, its kind and the source record it comes from, and links to the drug's ChEMBL compound page or FDA UNII record. The snapshot needs no setting and runs with or without the drug names add-on. To get it, write to support@conductscience.com, then load it as any snapshot (Load a copy). The sets then show under Search in, after the others. An older version of the service, which does not know them, leaves the snapshot out and logs why.
The page never checks the drug sets for you, because their records carry their sources' own terms. The snapshot holds ChEMBL's data, so it is shared under CC BY-SA 3.0, as the drug names folder is. Its ATTRIBUTION.txt credits ChEMBL and GSRS, and Sources and terms has a card for it. Its records give no public date, so their hits show none, and case plans do not search them. For a newer release of the drug chains, load its snapshot: each set is searched from the newest snapshot that holds it, and a saved search keeps the snapshot it ran on.
Claimed sequences
Claimed sequences is an add-on: it runs only when BIOSEQUENCE_ADDONS lists claimed. For each US application or patent of a hit, it gives the claims of that publication that name the hit's SEQ ID NO, and whether one of them is independent. Fixed rules read them from the claims in the US patent office's weekly full-text files, into one folder that ConductScience builds from those files. Without the add-on, the service reads no such folder, and new searches show no claims.
A search reads the claims as it runs, and its job keeps them, so it shows the same claims after the folder is replaced or the add-on is turned off. A search from before the add-on ran shows none until it is run again. Hits of an attached database show none.
The folder is read where it is, and not loaded into the data volume: its claims file is large, and the service reads only the rows a search needs. To get it, write to support@conductscience.com. Put it on the server, make it read-only, and run sha256sum -c SHA256SUMS in it, which checks every file. Then mount it into the container read-only:
-v /srv/sequences/claimed/<folder>:/claimed:ro
Add claimed to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_CLAIMED=/claimed, then remove the container and start it again. At start the service checks the folder's manifest and its SHA256SUMS, and reads every file but the claims file, whose size it checks. On several servers, give each worker the same mount and settings, since workers run the searches. The log then says Claimed sequences from <folder>: the claims of <count> US publications, weeks of <first> to <last>, rules <version>., and Sources and terms credits the US patent office. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer folder, mount it in place of the old one, then remove the container and start it again. To stop showing claims, take claimed out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_CLAIMED, then remove the container and start it again.
Patent families
Patent families is an add-on: it runs only when BIOSEQUENCE_ADDONS lists families. For each patent publication of a hit, it gives the family that Google Patents Public Data places the publication in, how many publications that family holds there and their offices, and the user guide's Patents list groups by those families first. ConductScience builds the folder from Google Patents Public Data. The folder starts in 2001: it places no earlier publication, and its counts and lists of a family's publications start that year. Without the add-on, the service reads no such folder, and families come from the user guide's rules alone.
A search reads the families as it runs, and its job keeps them, so it shows the same families after the folder is replaced or the add-on is turned off. A family's pane reads the family's other publications from the folder when it opens, so it lists none while the add-on is off. A search from before the add-on ran groups by the rules alone until it is run again.
The folder is read where it is, and not loaded into the data volume: its members file is large, and the service reads only the rows a pane needs. To get it, write to support@conductscience.com. Put it on the server, make it read-only, and run sha256sum -c SHA256SUMS in it, which checks every file. Then mount it into the container read-only:
-v /srv/sequences/families/<folder>:/families:ro
Add families to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_FAMILIES=/families, then remove the container and start it again. At start the service checks the folder's manifest and its SHA256SUMS, holds the families file in memory and checks the members file's size. On several servers, give each worker the same mount and settings, since workers run the searches. The log then says Patent families from <folder>: <count> publications placed in families, from Google Patents Public Data of <date>, rules <version>., and Sources and terms credits Google Patents Public Data. If the folder is missing or fails its checks, the service does not start, and the log says why. For a newer folder, mount it in place of the old one, then remove the container and start it again. To stop grouping by these families, take families out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_FAMILIES, then remove the container and start it again.
Patent status
Patent status is an add-on: it runs only when BIOSEQUENCE_ADDONS lists status. For each US application or patent of a hit, it gives the application's status in the US patent office's words, the office's status date, and a group: Granted, Pending, Expired (fee unpaid), Abandoned or Other. The statuses come from the office's Open Data Portal, copied on one day into one folder that ConductScience builds. Without the add-on, the service reads no such folder, and new searches show no status.
A search reads the statuses as it runs, and its job keeps them, so it shows the same statuses after the folder is replaced or the add-on is turned off. A search from before the add-on ran shows none until it is run again. Hits of an attached database show none. The office keeps Patented Case after a patent's term ends, so Granted does not mean a patent is in force, and the service does not estimate when a term ends.
The folder is read where it is, and not loaded into the data volume. The service holds its two tables in memory: the first folder adds under 200 MB. To get a folder, write to support@conductscience.com. Put it on the server, make it read-only, and run sha256sum -c SHA256SUMS in it, which checks every file. Then mount it into the container read-only:
-v /srv/sequences/status/<folder>:/status:ro
Add status to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_STATUS=/status, then remove the container and start it again. At start the service checks the folder's manifest and its SHA256SUMS, and reads every file. On several servers, give each worker the same mount and settings, since workers run the searches. The log then says Patent status from <folder>: the status of <count> US applications for <count> US publications, read on <day>, grouping rules <version>., where the grouping rules are the table in the user guide's Patent status section, and Sources and terms credits the US patent office. If the folder is missing or fails its checks, the service does not start, and the log says why and what to do. For a newer folder, mount it in place of the old one, then remove the container and start it again. To stop showing statuses, take status out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_STATUS, then remove the container and start it again.
Modifications
Modifications is an add-on: it runs only when BIOSEQUENCE_ADDONS lists modifications. For each hit, it gives the residues and bases the hit's records mark as modified, with their positions and the records' own words. They come from the modification features of GenBank's patent sequences, EMBL-EBI's US patent proteins and the US patent office's sequence listings, copied on one day into one folder that ConductScience builds. Fixed rules pick the features that mark a modified residue or base. They leave out a note that only says a position may be any of several residues or bases, such as "Lys or Asn", and the log names the rules' version. Without the add-on, the service reads no such folder, and new searches show no modifications.
A search reads the modifications as it runs, and its job keeps them, so it shows the same modifications after the folder is replaced or the add-on is turned off. A search from before the add-on ran shows none until it is run again. Hits of an attached database show none.
The folder is read where it is, and not loaded into the data volume. The service holds the folder's table in memory, which takes up to twice the size of records.tsv, and up to three times while it reads the folder at start. To get a folder, write to support@conductscience.com. Put it on the server, make it read-only, and run sha256sum -c SHA256SUMS in it, which checks every file. Then mount it into the container read-only:
-v /srv/sequences/modifications/<folder>:/modifications:ro
Add modifications to BIOSEQUENCE_ADDONS and set BIOSEQUENCE_MODIFICATIONS=/modifications, then remove the container and start it again. At start the service checks the folder's manifest and its SHA256SUMS, and reads every file. On several servers, give each worker the same mount and settings, since workers run the searches, and each worker holds its own copy of the table in memory. The log then says Modifications from <folder>: <count> modified residues and bases on <count> patent sequence records, read on <day>, rules <version>., and Sources and terms credits the three sources. If the folder is missing or fails its checks, the service does not start, and the log says why and what to do. For a newer folder, mount it in place of the old one, then remove the container and start it again. To stop showing modifications, take modifications out of BIOSEQUENCE_ADDONS and remove BIOSEQUENCE_MODIFICATIONS, then remove the container and start it again.
Export formats
Six exports save rows: a search's Alignments tab (matches) and Patents tab (patents), a combined search's references (combined), a case's References step by feature (case-references) and under All references, each reference once (case-all-references), and a case's claim charts, one row per cell (case-charts). Each has a built-in format, the CSV file the user guide describes. A file of your own adds formats. Each picks columns from its export's fields, in your order and under your headers, writes CSV, TSV or JSON lines, and can leave out or mask fields. People choose a format under Format, beside the export button, and the page says how many rows the file holds and what it left out.
Write the file as JSON with one key, formats, a list of 1 to 100 formats, put it beside the license in /etc/sequences, and set BIOSEQUENCE_EXPORT_FORMATS_FILE=/config/export-formats.json. Then remove the container and start it again. Each format takes:
| Key | What it holds |
|---|---|
name | 1 to 60 letters, digits, spaces and . _ ( ) & + , / -, shown under Format and in each record of an export. Each format has its own name, and none may be Built-in |
version | MAJOR.MINOR.PATCH, such as 1.0.0 |
audience | internal or external. An external format always leaves out the rows of confidential sets |
export | The export it serves: matches, patents, combined, case-references, case-all-references or case-charts |
type | csv (the default), tsv or jsonl |
columns | The fields to write, in order: a field's name, or { "field": "...", "header": "..." } for a header of 1 to 100 characters. Without it, every field in the built-in order |
redact | { "omit": [...], "mask": [...] }: the fields left out of the file, and the fields whose every cell reads [redacted] |
The fields of each export, in the built-in order:
matches:query,subject,title,engine,strand,mark,mark_reason,accession,first_published,offices,publications,query_match,identity,query_identity,coverage,evalue,bits,score,exact_tool,exact_score,exact_identity,exact_query_match,exact_coverage,exact_matches,exact_gaps,exact_align_length,align_length,matches,mismatches,gaps,query_from,query_to,subject_from,subject_to,subject_length,occurrences,positions,reached_limit,entries,papers,pubmed_ids,dois,paper_dates,paper_marks,first_publication,first_kind,first_applicant,first_filed,first_application_number,first_priority,changes,substitutions,insertions,deletions,wildcard_matches,x_letters,identity_x_as_match,known_changes,query_changes,clinvar_variants,species,taxa,species_groups,drugs,drug_sources,genes,gene_ids,gene_links,claimed,claimed_in,hit_identity,patent_status,patent_status_group,patent_status_as_of,stored_codes,modifications,modification_positionspatents:office,number,kind,published,applicant,title,mark,mark_reason,sequences_hit,seq_nos,best_query_match,best_identity,hits,link,undated,after_date,application_number,filed,priority,family,family_size,family_basis,date_group,why_listed,claimed_seq_nos,not_claimed_seq_nos,patent_status,patent_status_date,patent_status_group,patent_status_as_of,family_id,family_publications,family_offices,modified_seq_noscombined:reference,id,office,number,kind,accession,source,published,applicant,title,mark,mark_reason,sets,queries,sequences_hit,seq_nos,best_query_match,best_identity,hits,best_set,best_query,best_job,best_row,link,undated,after_date,application_number,filed,priority,family,family_size,family_basis,date_groupcase-references:feature,claim,claim_words,threshold,rank,meets,coverage,date_class,query_match,evalue,reference,id,label,office,number,kind,accession,source,published,filed,priority,application_number,applicant,title,link,matched,meeting,of,claimed_seq_ids,reference_seq_nos,best_search,best_set,best_job,best_row,best_query,best_seq_id_no,query_identity,identity,bits,align_length,matches,searches,sets,mark,marks,meets_words,date_words,matched_wordscase-all-references:rank,best_rank,features_met,features,reference,id,label,office,number,kind,accession,source,published,filed,priority,application_number,applicant,title,link,date_class,family,family_size,family_basis,feature_ranks,feature_meets,feature_best,feature_marks,mark,searches,sets,files,date_wordscase-charts:claim,row,row_kind,features,claim_words,description,column,reference,label,published,filed,date_words,link,kept,state,rule_state,rule_facts,sources,answer_state,answer_note,answer_user,answer_at,rule_then,rules_version
A CSV or TSV cell that starts with =, +, - or @ gets a ' before it, as in the built-in CSV, so a spreadsheet does not run it. A TSV cell writes a tab, a line break and a backslash as \t, \n and \\. A JSON lines file holds one object per row, keyed by the headers.
For example, this format gives your docketing team the patents of a search as TSV, with its own headers:
{
"formats": [
{
"name": "Docketing patents",
"version": "1.0.0",
"audience": "internal",
"export": "patents",
"type": "tsv",
"columns": [
{ "field": "office", "header": "Office" },
{ "field": "number", "header": "Number" },
{ "field": "kind", "header": "Kind" },
{ "field": "published", "header": "Published" },
{ "field": "applicant", "header": "Applicant" },
{ "field": "family", "header": "Family" }
]
}
]
}
And this one gives outside counsel the references of a combined search as CSV: no titles or notes, the applicant masked, and none of the rows a confidential set found:
{
"formats": [
{
"name": "Counsel references",
"version": "1.0.0",
"audience": "external",
"export": "combined",
"columns": ["id", "published", "applicant", "sets", "best_query_match", "best_identity", "title", "mark_reason"],
"redact": { "omit": ["title", "mark_reason"], "mask": ["applicant"] }
}
]
}
At start the service checks the file and stops with a message that names the format and the problem. It keeps the SHA-256 of each format, by its name and version, in export-formats.json in the data volume. A format whose content changed while its name and version stayed the same stops the start: give a changed format a new version, so that each record names the format that wrote its file. The log names the formats it read. The built-in formats take the release's version.
Each export is written to a file the service holds under work/exports/ in the data volume while it hashes it, then sent and deleted. Its record names the export, the format's name, version, audience, type and SHA-256, the file's SHA-256 and size, its rows, the rows left out because a confidential set found them, and the fields left out or masked. It never holds a value from the file. A search keeps these records in exports.jsonl in its job folder, and programs read them at GET /api/jobs/{id}/exports. A combined search keeps them in its record, and a case in its log, as export.made events, which the audit export carries. The file's response headers say the same: X-Export-Format, X-Export-Format-Version, X-Export-Format-Sha256, X-Export-Sha256, X-Export-Rows, X-Redacted-Rows and X-Redacted-Fields.
Confidential sets
BIOSEQUENCE_CONFIDENTIAL_SETS is a comma-separated list of what holds confidential data, such as unpublished applications you attach as a database. Give each snapshot set by its name, such as patent_nucl, each source of a snapshot's records by its ID, such as ebi-epo, and each attached database by the path it was attached from or by its ID. Every external format leaves out the rows that a named set or database found, even when other sets found them too, and the rows with a record from a named source. No setting turns that off. Internal formats and the built-in formats keep every row. A name the service does not know as a set, a source, an ID or a full path stops the start.
Working files
Working files are what Sequences makes while it works and that no record keeps. The service deletes them as soon as their work ends:
| Working file | Where | Deleted |
|---|---|---|
| A sequence listing, or a result file imported into a case, still arriving | listings/<id>/upload.bin, cases/<id>/imports/ | Once all of it has come and its import job has read it, or when its session ends first |
| An export, held while it is written and sent | work/exports/ | Once it is sent, or when its session ends first |
| A search's query in parts and each part's output, a scan's parts, FASTA36's list of library files, the JSON a search of an attached database is read from, and a file still being written | jobs/<id>/ | When the search finishes, fails or is cancelled |
| An exact alignment's library, query and FASTA36 output, and a multiple alignment's input and MAFFT output | jobs/<id>/ | When it finishes, fails or is cancelled. The alignment itself stays |
| Everything a job that failed, was cancelled or was cut off made, but its job and its request | jobs/<id>/ | Then. Run again reads the request |
| A pairwise alignment or an antibody numbering in progress | work/<id>/ | When it answers |
| What the page keeps in the browser, such as a closed Things to try | The browser | When the person signs out |
With sign-in on, each upload and export belongs to the browser session that started it. When the person signs out, the service stops that session's uploads and exports and deletes their files. Another browser's session, of the same account or another, keeps its own. When a sign-in ends without a sign-out, the service does the same within a minute. A sign-in made again in the same browser before the old one ends keeps the session and its files. Signing out deletes no record: searches, listings, cases and the rest stay.
At each start the service deletes what a stop left: the scratch folder work/, the working files of jobs that had reached their final state, and the files of sessions that were sending or receiving. The first start of a release that does this also deletes the working files earlier releases kept in job folders. The audit log counts each deletion; see Audit export for a SIEM.
How long records are kept
Records stay until their owners delete them, unless BIOSEQUENCE_RETENTION_DAYS is set. Set to a number of days from 1 to 36,500, it removes these once they are that many days old, counted from when they finished:
- Searches that no case holds, with their exact alignments.
- Portfolio exports.
- Combined searches and batches, each whole once all its searches are that old and no case holds any of them.
A search a case holds stays, and so do the search a watch follows and the watch's latest run. Listings, saved sequences, watches, multiple alignments, attached databases and cases stay: people delete them on the page, and an admin deletes a case as below. The service looks when it starts and then every hour, and the audit log notes each look that removed something, with the count of each kind and their bytes. The usage page counts the work still on the server, so a search that was removed no longer counts.
An admin deletes a case with its searches through the API, with an admin's API token (see API tokens for programs):
curl -X DELETE -H "Authorization: Bearer <token>" "https://sequences.example.org/api/admin/cases/<case ID>"
It deletes the case's claims, plan, runs, decisions, notes, reports, log and imported result files, every search its plan ran, and each search added to it that no other case holds. A search still running is cancelled first, and a case the claims model is reading answers 409 until the model has answered. Listings stay. A deletion cannot be undone. The audit log keeps a case.deleted event with the case's ID, its owner, the number of events in its log and the SHA-256 of the last one, the count of searches deleted and their bytes. Only the people in BIOSEQUENCE_ADMINS can delete a case, and with sign-in off the address does not exist.
Upgrades
The data volume and /etc/sequences stay as they are. Only the image changes. Back up first (see Backups), and upgrade when no searches are running, because stopping the service cancels them:
docker load -i sequences-<new version>.tar
docker run --rm conductscience/sequences:<new version> versions
docker stop sequences
docker rm sequences
Then start it with the same docker run command as before and the new version. Keep the previous image until you have checked the new one. The release notes list what each release changed.
Searches that docker stop cancels show as cancelled, and people can start them again. Searches a program sent in a batch are cancelled the same way, and the batch shows them as cancelled, so the program can send those again.
With several servers, upgrade them all together, since a coordinator refuses a worker of another release. Load the new image on every host, then:
- Drain each worker, then stop and remove it. See Drain and stop a worker.
- Stop and remove the coordinator.
- Start the coordinator with the new version, then the workers.
The coordinator's searches of a snapshot keep their place through the upgrade, and one it ran itself starts again. The workers take them once they join. Its other work is cancelled at its stop, as on a single server.
Only one service may use a data volume at a time, and it holds a lock (.lock) on the volume while it runs. A service that stops cleanly releases the lock at once. After a crash, the old container's lock expires 30 seconds after its last renewal. A new container waits for that, up to 50 seconds, then starts.
Backups
The data volume holds:
| Path in the volume | What it is | Back up? |
|---|---|---|
jobs/ | Every search and alignment, with its results, marks and record, each portfolio export's CSV and FASTA files, and each result file imported into a case, as it came, with its joined rows | Yes |
databases/ | FASTA collections imported on the page with sign-in off. An import that fails, is cancelled or is cut off by a stop leaves no folder | Yes |
listings/, fragments.jsonl, query-groups.jsonl | Uploaded sequence listings, saved sequence fragments and saved query groups | Yes |
cases/ | Cases: their claims, search plans, searches added by hand or imported, notes and the log of every step. Reports are not kept, and the log has each one's SHA-256 | Yes |
watches/ | Watches: each one's search or applicant, the matches or publications it has seen and its runs, and the key behind the stop links in watch emails | Yes |
tokens.json, batches/ | API tokens (a hash of each, never the token itself, with its name, account and state) and each batch's list of searches | Yes |
applicants/ | Each account's applicant joins and the log of every join and undo | Yes |
audit/ | The service's audit log: sign-ins, API tokens, starts and their settings, invite file changes, cases handed to another account, deletions and audit exports. See Audit export for a SIEM | Yes |
papers/, ror/, taxa/ | Paper folders, ROR name indexes and taxa folders, read-only | Yes. The service does not start without the folders its settings name, and a paper folder made again gets a new ID, and a taxa folder a new build time |
taxa-scans/ | The source scans a taxa folder was made from | Yes: once the source files are gone, a scan cannot be made again, and a new taxa folder needs it |
clinical/ | Clinical context folders, read-only, with clinical context on | Optional if you keep the copies you received: load makes the folders again from them |
funding/ | Funding trail folders, read-only, with the funding trail on | Optional if you keep the copies you received: load makes the folders again from them |
drugs/ | Drug names folders, read-only, with drug names on | Optional if you keep the copies you received: load makes the folders again from them |
accounts.json | The name each account last signed in with, for the usage page and for handing a case to a colleague, and the group its last sign-in found when sign-in requires one | Yes |
sign-in-provider.json | The sign-in provider the volume belongs to. See Move to another provider | Yes |
session-secret, calibration.json | The cookie key and the run-time estimates. A search whose estimate cannot be written, on a full disk say, still completes, and the log says so | Yes |
export-formats.json | The SHA-256 of each export format the service has read, by name and version. See Export formats | Yes |
corpora/, deltas/ | Snapshots and deltas, read-only | No, if you keep the copies you loaded: load them again from those. A snapshot you built yourself cannot be built again from the same files, since GenBank keeps only its current release, so back it up as below |
checked/ | The SHA-256 list of each snapshot and delta, which a coordinator and its workers compare | Optional: load writes it, and without it a coordinator or worker reads each folder whole at its next start |
inputs/, work/, sessions/, .lock | Source files for a build, scratch space, the record of each signed-in session, and the running service's claim | No |
Each file on the data volume is readable only by the service's account. On a server you run, protection at rest comes from your environment, such as an encrypted volume, for the data volume and its backups alike.
With several servers, back up the coordinator's volume: it holds every search, result, case and log. A worker's volume holds its copies of the snapshots, deltas, paper folder and taxa folder, which you load or copy again, and the searches it runs, which start over on another server when it stops.
Back up /etc/sequences too: the license, the settings and the client secret. On the server:
sudo install -d -m 0700 /srv/backup
sudo tar -czf /srv/backup/sequences-config-$(date +%F).tar.gz -C /etc sequences
The command below copies the data volume, all but the snapshots, deltas, scratch files and session records. To keep snapshots and deltas you built yourself, leave out --exclude=./corpora --exclude=./deltas. Stop the service first with docker stop sequences, which cancels any search still running, and start it again afterwards with docker start sequences. The one-off container runs tar as root so it can write to your backup folder. The archive holds the cookie key, so keep it as safe as the server:
docker run --rm --user 0 \
-v sequences-data:/data:ro \
-v /srv/backup:/backup \
--entrypoint tar conductscience/sequences:<version> \
--exclude=./corpora --exclude=./deltas --exclude=./inputs --exclude=./work --exclude=./sessions --exclude=./.lock \
-czf /backup/sequences-data-$(date +%F).tar.gz -C /data .
To restore on a new server, or after you remove the old volume (docker volume rm sequences-data), put the settings back, create the volume and unpack the archive. Then load the same snapshots and deltas again, unless the archive holds them, so older results can open their records:
sudo tar -xzpf /srv/backup/sequences-config-<date>.tar.gz -C /etc
docker volume create sequences-data
docker run --rm --user 0 \
-v sequences-data:/data \
-v /srv/backup:/backup:ro \
--entrypoint tar conductscience/sequences:<version> \
-xzpf /backup/sequences-data-<date>.tar.gz -C /data
Measured on release 0.92.0, with 2 CPUs for each command, on a coordinator's volume of 3.4 GB without its snapshots that held 580 searches: the backup took 61 seconds and made a 339 MB archive, and the restore took 17 seconds. Every file of the restored volume matched the original by SHA-256, size, permissions and time, its audit log checked whole, and audit-export gave the same events from both.
Network traffic
What the container receives:
- Sign-in off: connections to port 8765 on the server's own
127.0.0.1, through host networking. - Sign-in on: connections from your reverse proxy, on the same server, to
127.0.0.1:8765. The service answers only requests that name an allowed host, andGET /api/healthat an IP address. With the AWS kit, its load balancer connects to the server's private address instead. See In your AWS account. - The service keeps an idle connection open for 65 seconds, and no setting changes that. A proxy or load balancer that reuses connections should close idle ones sooner, as an AWS load balancer does after 60 seconds.
- A coordinator: its workers' connections to the worker port, 8766 by default, on
127.0.0.1inside its container unlessBIOSEQUENCE_WORKER_BINDsays otherwise. It takes only requests that carry the worker key. A worker receives nothing: it opens no port. See Several servers.
What the container sends while it runs:
- Sign-in off: nothing, unless you set a claims model (below). With sign-in off the service sends no watch email.
- Sign-in on: requests to your sign-in provider, over HTTPS. The discovery document and the provider's public keys, each kept for an hour and fetched again early when a token names a key the service has not seen; the token endpoint, once per sign-in; and the userinfo endpoint, only when the ID token leaves out a claim the service needs.
- The search programs run on the snapshot in the data volume. BLAST+ can send usage reports to NCBI, and the service turns that off (
BLAST_USAGE_REPORT=false) for every program it starts. - Claim suggestions from a language model are off unless you set
BIOSEQUENCE_CLAIMS_MODEL_URLandBIOSEQUENCE_CLAIMS_MODEL. With them set, a case's claims, with the paragraphs of its description chosen for them, go to that address when someone asks for suggestions. The address may usehttp://only on127.0.0.1orlocalhost. Any other host must be named inBIOSEQUENCE_CLAIMS_MODEL_HOSTSand usehttps://. With sign-in on,127.0.0.1is the container itself, so a model on the same server needs a container that shares that address (--network container:sequences) or a named host with HTTPS. See Claim suggestions from a model. - Watch emails are off unless you set
BIOSEQUENCE_MAIL_FROMandBIOSEQUENCE_RESEND_KEY. With them set, the service posts each email tohttps://api.resend.com/emailsover HTTPS: the person's address, how many new matches or publications one of their watches found, and links back to the service. WithBIOSEQUENCE_MAIL_DETAILS=on, it also posts the watch's name and its new matches or publications. - Patent context reads only its folder in the data volume and makes no network connections.
- Clinical context, the funding trail, drug names and antibody names, when they run, read only their folders in the data volume and make no network connections. Nor do the dates from Crossref: the paper folder holds them, and a search asks Crossref nothing.
- The license is checked inside the container against the key built into the image. There is no license server, and the service sends no usage data anywhere.
- A worker: requests to its coordinator's worker port, over HTTPS between hosts, and nothing else. It asks for searches, sends their progress and results back, and says hello every 20 seconds. A coordinator sends nothing to its workers but its answers.
What the browser loads: the page, its scripts, styles and font, all from the service itself. Links leave it only when someone clicks them: the patent link on each result and on each new publication a watch lists (BIOSEQUENCE_PATENT_LINK), a patent's links to Espacenet and, for a US publication, to Patent Public Search, the PubMed and DOI links of the papers a hit cites and of their notices, the terms pages under Sources and terms, with BIOSEQUENCE_ROR set, the link to a suggested ROR record (https://ror.org/<id>), with BIOSEQUENCE_ANTIBODY_NAMES set, the link to each named antibody's page at ChEMBL (https://www.ebi.ac.uk/chembl) and to the names' license at Creative Commons, and, with clinical context on, the links to Open Targets (https://platform.opentargets.org), ChEMBL (https://www.ebi.ac.uk/chembl) and ClinicalTrials.gov (https://clinicaltrials.gov). With the funding trail on, the page also links to NIH RePORTER (https://reporter.nih.gov). With drug names on, it links each named drug to its ChEMBL compound page (https://www.ebi.ac.uk/chembl) or its FDA UNII record (https://precision.fda.gov).
Commands you run yourself: fetch downloads the public source files from NCBI, EBI and UniProt over HTTPS, and crossref asks https://api.crossref.org for the dates of papers PubMed does not hold. antibody-names fetch downloads a ChEMBL release and its molecules' names from EBI over HTTPS. build, delta, load, verify, sums, attribution, papers, ror, antibody-names build, context extract, context assemble, context check and check-license make no network connections. context ops, only with BIOSEQUENCE_OPS=on, asks EPO's Open Patent Services at ops.epo.org for the numbers of the snapshot's publications of other offices. To hold the service to this, allow its container outbound HTTPS to your provider's host names, to your claims model's host if you set one, to api.resend.com if you turn on watch emails, and nothing else. A container that runs fetch also needs outbound HTTPS to ftp.ncbi.nlm.nih.gov, ftp.ebi.ac.uk and ftp.uniprot.org, one that runs crossref to api.crossref.org, one that runs antibody-names fetch to ftp.ebi.ac.uk and www.ebi.ac.uk, and one that runs context ops to ops.epo.org. A worker needs only its coordinator's worker port, and the coordinator's worker port only the workers' addresses.
Questions about installing Sequences go to support@conductscience.com.
Report a vulnerability
To report a security vulnerability in Sequences, write to security@conductscience.com. Say which release you run, as /docs shows it, what you found and how to see it again. ConductScience reads that address at any hour. It fixes a critical vulnerability within 7 days and a high one within 30 days, and the release notes name each fix. Please give it that time before you tell others.
Every server answers /.well-known/security.txt with that address and a link to this section, in the standard form that security tools read. To send reports about your own server to your own security team, answer that address at your reverse proxy instead.