Page version 1.1.0
Benchmarks
How Sequences did in recorded tests, and what each result means for a patent search. The tests compare the search methods Sequences runs, BLAST among them.
What the tests found
The benchmark behind most of these answers searched 28,757,943 DNA sequences and 10,491,135 proteins from patents. Each question below names the release of Sequences it was tested on, and the release notes say what each later release changed.
Does it count every exact match?
Yes105 of 105
searches counted the same matches as an independent text search
Does it find short DNA, such as primers?
Yes132,309 of 132,309
patent sequences found that hold 72 short DNA fragments
BLAST alone found 57.1%
Does it find close variants?
Yes60 of 60
protein fragments found at 90% identity
BLAST found 53 of 60
Does it find distant variants?
Mostly39 of 60
protein fragments found at 40% identity
BLAST found 33 of 60
Do I get the same answer twice?
Yes1,173 of 1,173
searches gave identical results in three runs
Does the listing check catch ST.26 errors?
Yes115 of 115
listings with ST.26 errors flagged
It also flagged 1 that validation passed, citing paragraph 39(a) of ST.26.
Which search fits your task
| Your task | Use |
|---|---|
| A primer, probe or other short DNA | Automatic, which adds an exact scan to BLAST for short DNADoes it find short DNA, such as primers? |
| A peptide or other sequence that must appear letter for letter | Exact textDoes it count every exact match? |
| A variant claim, such as at least 90% identical to a SEQ ID NO | Smith-Waterman, which Automatic runs for proteinDoes it find variants of a sequence? |
| A claim that allows a set number of changed letters | Within k changesDoes it find every near match? |
| A search you may have to defend | Any method: each search keeps a record of the versions, data and settings it ran withDo I get the same answer twice? |
| A sequence listing, before you file it | The listing checkDoes the listing check catch ST.26 errors? |
For every method, see Choosing a search method, and for how long each takes, Searches measured on the hosted service.
Does it count every exact match?
Yes In 105 of 105 searches, Sequences counted the same matches as GNU grep, an independent text search of the same sequence files.
When a claim recites a peptide or a primer, a prior-art or freedom-to-operate search has to list every patent sequence that holds it.
How it ran, with every number
Exact text and Pattern count every stored sequence that holds a query. The benchmark of Sequences 0.22.0, run on November 6, 2026, checked each count against GNU grep, which read the same sequence files and shares no code with Sequences.
Data: snapshot 2026-11-05-c456f6bdba29 of the patent sets, with 28,757,943 DNA sequences of 15,612,493,945 bases in all and 10,491,135 proteins of 1,718,205,235 residues.
| Queries | Searches | Counted by Sequences | Counted by GNU grep | Equal |
|---|---|---|---|---|
| DNA fragments of 15 to 50 bases | 72 | 145,561 | 145,561 | 72 of 72 |
| Peptides of 8 to 30 residues | 20 | 317 | 317 | 20 of 20 |
| DNA patterns | 7 | 24,221,001 | 24,221,001 | 7 of 7 |
| Protein patterns | 6 | 6,360,246 | 6,360,246 | 6 of 6 |
105 of the 105 counts were equal, query by query. A match is a stored sequence that holds the query, counted once for each DNA strand that holds it.
Evidence: biosequence-benchmark-0.22.0.json, version 2.0.0.
Does it find short DNA, such as primers?
Yes Automatic listed all 132,309 patent sequences that hold the 72 short DNA fragments it searched, 15 to 50 bases long. BLAST alone listed 75,570, 57.1%: Sequences runs it with a limit of 5,000 sequences for each query, and 41 of the 72 fragments reached that limit.
Claims often recite primers and probes. A prior-art check for one has to find every patent sequence that holds it, however many there are.
Table view
| Length | GNU grep | Exact text | BLAST |
|---|---|---|---|
| 15 bases | 38,142 | 38,142 (100%) | 18,878 (49.5%) |
| 18 bases | 10,097 | 10,097 (100%) | 9,658 (95.7%) |
| 21 bases | 14,019 | 14,019 (100%) | 13,590 (96.9%) |
| 25 bases | 9,693 | 9,693 (100%) | 9,624 (99.3%) |
| 30 bases | 51,469 | 51,469 (100%) | 14,931 (29.0%) |
| 50 bases | 8,889 | 8,889 (100%) | 8,889 (100%) |
| All | 132,309 | 132,309 (100%) | 75,570 (57.1%) |
How it ran, with every number
The benchmark of Sequences 0.22.0 cut 72 fragments of 15 to 50 bases from stored patent DNA, unchanged, 12 of each length. Each was searched with Exact text, which scans every stored sequence, and with BLAST, and GNU grep listed every stored sequence that holds it.
| Length | Exact text | BLAST |
|---|---|---|
| 15 bases | 12 of 12 | 10 of 12 |
| 18 bases | 12 of 12 | 12 of 12 |
| 21 bases | 12 of 12 | 12 of 12 |
| 25 bases | 12 of 12 | 12 of 12 |
| 30 bases | 12 of 12 | 11 of 12 |
| 50 bases | 12 of 12 | 12 of 12 |
| All | 72 of 72 | 69 of 72 |
Exact text listed every sequence GNU grep found, 132,309 of 132,309. BLAST listed 75,570: it listed at most 5,000 sequences for each query, and 41 of the 72 fragments reached that limit. BLAST also did not return the stored sequence that 3 of the fragments were cut from. Automatic, which adds an exact scan for short DNA, listed all 132,309.
Evidence: biosequence-benchmark-0.22.0.json, version 2.0.0.
Does it find variants of a sequence?
Mostly Smith-Waterman, which Automatic runs for protein, found 60 of 60 protein fragments that were 90% identical to their source, and 39 of 60 that were 40% identical. At every level of identity it found more than BLAST.
A variant claim reaches sequences that differ from the one it recites, such as those at least 90% identical to it. A prior-art or freedom-to-operate search has to find them too.
Table view
| Identity | Smith-Waterman | BLAST | Exact text |
|---|---|---|---|
| 100% | 60 | 54 | 60 |
| 95% | 60 | 54 | 6 |
| 90% | 60 | 53 | 0 |
| 80% | 58 | 51 | 0 |
| 70% | 58 | 49 | 0 |
| 60% | 55 | 44 | 0 |
| 50% | 48 | 40 | 0 |
| 40% | 39 | 33 | 0 |
How it ran, with every number
The benchmark cut peptides and proteins of 8 to 500 residues from stored patent proteins and changed letters at random, keeping 100% down to 40% of them, which is each query's identity to the sequence it was cut from, 60 queries at each level. A query counts as found when the stored sequence it was cut from comes back. For proteins, Smith-Waterman ran as the automatic choice.
In all, Smith-Waterman found the most of these queries, then BLAST, then Exact text, and the other altered sets gave the same order:
| Queries | Searched | Smith-Waterman | BLAST | Exact text |
|---|---|---|---|---|
| Proteins, letters changed at random, 100% to 40% kept | 480 | 438 | 378 | 66 |
| Proteins, letters changed as BLOSUM62 weighs them, 95% to 40% kept | 210 | 192 | 159 | 3 |
| Proteins with 2 to 10 insertions or deletions per 100 residues | 90 | 83 | 74 | 0 |
| DNA fragments of 21 to 50 bases, 95% to 80% kept | 60 | 53 | 47 | 0 |
Evidence: biosequence-benchmark-0.22.0.json, version 2.0.0.
Does it find every near match?
Yes Within k changes found all 20 changed DNA fragments it searched, among them the 7 of 60 that Smith-Waterman missed in the benchmark.
Some claims allow a set number of changed letters. Within k changes lists every patent sequence within that number, with no score cutoff, so its counts are complete.
How it ran, with every number
In the benchmark, Smith-Waterman missed 7 of the 60 altered DNA fragments. Within k changes lists every stored sequence that differs from a query by at most k changes, with no score or E-value cutoff. On November 15, 2026, in the test runs before its release in Sequences 0.95.0, it searched all 20 fragments of the 4 groups that held those misses: first counting substitutions only, then insertions and deletions too.
| Fragments | Letters changed | Smith-Waterman | Within k changes |
|---|---|---|---|
| 21 bases, 90% kept | 2 | 4 of 5 | 5 of 5 |
| 21 bases, 80% kept | 4 | 1 of 5 | 5 of 5 |
| 30 bases, 95% kept | 2 | 4 of 5 | 5 of 5 |
| 50 bases, 95% kept | 3 | 4 of 5 | 5 of 5 |
| Fragments | Substitutions only | Insertions and deletions too |
|---|---|---|
| 21 bases, 90% kept | 2 | 2 |
| 21 bases, 80% kept | 4 | 3 to 4 |
| 30 bases, 95% kept | 2 | 2 |
| 50 bases, 95% kept | 3 | 3 |
No fragment needed a k larger than its changed letters, and k = 4 was the most any needed. Across the benchmark's queries with up to 5 letters changed, Within k changes found the stored sequence each was cut from at k equal to its changes: 186 of 186 protein queries and 117 of 117 DNA queries.
Evidence: 2026-11-15-within-k-changes-release, version 1.0.0; 2026-11-15-pilot-within-k-changes/results/golines.json, version 1.0.0; biosequence-benchmark-0.22.0.json, version 2.0.0.
Do I get the same answer twice?
Yes The same 1,173 Smith-Waterman and global searches, run three times, gave the same hits, scores and E-values each time. The benchmark also ran every query twice with each method, and each search found, or missed, its source sequence both times.
A search you rely on before the patent office or in court has to give the same result when someone runs it again.
How it ran, with every number
The benchmark of Sequences 0.22.0 ran every query twice. All 5,182 results, one for each query and method, were the same in both runs: whether the source sequence came back, every Exact text and Pattern count, and how many of the sequences GNU grep found each listed. But 265 of its 1,173 Smith-Waterman and global searches listed a different number of sequences, by up to 1,119, because the sample of scores behind their E-values could change from run to run.
Sequences 0.95.0 ships a build of FASTA36, the program behind those methods, that draws the same sample on every run. On November 15, 2026, that build ran the same 1,173 searches three times: runs A and B on 8 threads each, and run C on 12.
| Runs | Searches | Count | Sequences | Rows |
|---|---|---|---|---|
| A and B | 1,173 | 0 | 0 | 0 |
| A and C | 1,173 | 0 | 0 | 0 |
| B and C | 1,173 | 0 | 0 | 0 |
Evidence: biosequence-benchmark-0.22.0.json, version 2.0.0; 2026-11-15-pilot-sw-repeatable, version 1.4.0.
Does the listing check catch ST.26 errors?
Yes The listing check flagged all 115 listings that ST.26 validation found errors in, and agreed with it on 182 of the 183 listings tested. They disagreed on one, whose first line, the XML declaration, does not state its character encoding: the check flags it, citing paragraph 39(a) of ST.26, and validation let it through.
The check runs before you file, so you can fix a listing before the patent office finds the problem.
How it ran, with every number
The listing check, on the Listings tab of Sequence tools, checks a WIPO ST.26 sequence listing before it is filed. From November 7 to 10, 2026, its results were compared with ST.26 validation results on 183 listings: a base listing with no errors, 20 listings with known problems, 161 that each change one thing in the base, and one real listing.
| Result | Listings |
|---|---|
| ST.26 validation found an error | 115 |
| ST.26 validation found no error | 68 |
| The listing check agreed: an error where validation found one, none where it found none | 182 |
| The two differed | 1 |
They differed on one listing, whose XML declaration has no encoding: validation reported no error, and the listing check reports one, because paragraph 39(a) of ST.26 gives the declaration with its encoding.
Evidence: the listing check's record of November 8, 2026, version 1.1.1.
How long does a whole patent take?
82.5 minutes for one patent's 1,034 claimed SEQ ID NOs, the 1,026 with a sequence in its listing, searched in Patent proteins and Swiss-Prot with Smith-Waterman. The plan expected 93 minutes.
A freedom-to-operate search may have to cover every SEQ ID NO a patent's claims recite. A search plan says how long that will take before it starts.
How it ran, with every number
A search plan turns the SEQ ID NOs a patent's claims recite into searches. On November 14, 2026, the hosted service, then running Sequences 0.85.0, ran a plan for US 12,780,932 B1, whose claims recite 1,034 SEQ ID NOs. Its sequence listing skips 8 of them, so the plan searched the other 1,026, all proteins, with Smith-Waterman on Patent proteins and Swiss-Prot: 2,052 queries of 152,040 letters in all, as 104 jobs, 2 at a time.
| Run | Searched | Jobs | Expected | Took |
|---|---|---|---|---|
| Run 1 | Patent proteins | 52 | 65 minutes | 75.9 minutes |
| Run 2 | Swiss-Prot | 52 | 28 minutes | 6.7 minutes |
| Both | Patent proteins and Swiss-Prot | 104 | 93 minutes | 82.5 minutes |
Both is the time from the first job's start to the last job's end, and each time is rounded to a tenth of a minute.
All 104 jobs completed, in 82.5 minutes against the 93 the plan expected. The total came in under the estimate; the split between the two runs did not match it.
Evidence: 2026-11-14-query-groups-pilot, version 1.0.0.
Do more servers run more searches?
Yes With 3 workers, a server ran 2.04 times as many searches an hour as it did alone: 741.5 against 363.6.
A team that runs many searches on its own server can add workers to clear a queue sooner.
Table view
| Servers | 20 queued | 100 queued |
|---|---|---|
| The server alone | 363.6 | Ran beside other work |
| 2 workers | 598.5 | 552.1 |
| 3 workers | 741.5 | 828.0 |
How it ran, with every number
A Sequences server can hand searches to workers, other servers that take searches from its queue. On November 15, 2026, Sequences 0.92.0 ran batches of 20 and 100 searches, each batch queued at once, on snapshot 2026-11-13-5e447c75c43f: on a server alone, then on a server with 2 and with 3 workers. Half of each batch were proteins searched with Smith-Waterman and half DNA searched with BLAST. Each search ran on 6 threads. Each worker had 6 CPUs of its own; the server alone had 8 and ran one search at a time.
With 20 searches queued, 2 and 3 workers ran 1.65 and 2.04 times as many searches an hour as the server alone. With 100 queued, 3 workers ran 1.50 times as many as 2. The batches on one worker, and the server alone's batch of 100, ran beside other work on the same computer, so the record leaves them out of these comparisons.
Evidence: 2026-11-15-scaling-measured, version 1.2.0.
Terms on this page
- Automatic
- The first method on the search form. It picks the method by the query: Smith-Waterman for protein, BLAST for DNA, and for short DNA an exact scan as well, so its exact matches are complete.
- BLAST
- NCBI's program for finding similar sequences, and one of the methods Sequences runs. It starts from short stretches that match exactly, which makes it fast. Sequences keeps up to 5,000 sequences for each BLAST query.
- E-value
- About how many hits this good a search of this size would find by chance alone, so the smaller it is, the less likely the match is chance.
- Exact text
- Finds the query letter for letter in every stored sequence.
- FASTA36
- The program behind Smith-Waterman and the global alignment methods.
- Freedom-to-operate search
- A search for patents whose claims could cover a product or process you plan to make, use or sell.
- Global alignment
- Aligns the whole query with each stored sequence, rather than only its best-matching stretch.
- GNU grep
- A standard text-search program. The benchmark ran it on the same sequence files to count matches independently; it shares no code with Sequences.
- Identity
- The share of places where two aligned sequences have the same letter. A variant claim may cover sequences at least 90% identical to one it recites.
- Pattern
- Finds every stored sequence that fits a pattern, such as a primer with places that can hold any of several bases. It reads every stored sequence, so its counts are complete.
- Peptide, residue and base
- A residue is one amino acid of a protein, a base is one letter of DNA, and a peptide is a short protein.
- Prior-art search
- A search for earlier publications, patents among them, that may already disclose what a claim covers.
- SEQ ID NO
- The number a sequence listing gives each sequence. Claims cite sequences by it.
- Smith-Waterman
- Aligns the query with every stored sequence and finds the best-matching stretch of each. It takes longer than BLAST, and in these tests it found more changed sequences.
- Snapshot
- A read-only copy of public sequence files, built on a known date.
- ST.26
- WIPO ST.26, the standard for the sequence listing filed with a patent application, as an XML file.
- Swiss-Prot
- UniProtKB/Swiss-Prot, the UniProt Consortium's public set of reviewed protein sequences.
- Within k changes
- Lists every stored sequence that differs from the query by at most k changes, counting substitutions only or insertions and deletions too, with no score or E-value cutoff, so its counts are complete.
- Worker
- Another server that takes searches from a Sequences server's queue.