/
Blast+

Blast+

Blast+ is a program for comparing biological sequence information, such as amino-acid or nucleotide sequences.

On this page:

Running Blast+ 

Blast+ is provided by system modules. To load the Blast+ module, use the following command:

$ module load blast/2.12.0--pl5262h3289130_0

NCBI Indexed Data bases

Standard NCBI indexed databases, as used by the Blast+ executables, are currently centrally installed in the directory /scratch/references/blastdb_update/blast-YYYY-MM-DD/db. Where 'blastdb-YYYY-MM-DD' is the date of the download of the database files. The Blast+ databases are downloaded every scheduled maintenance period. To check the date of a given nucleotide or protein database, use the Blast+ utility blastdbcmd, which gives the date the file was updated in the central NCBI repository (via the anonymous FTP download site at ftp://ftp.ncbi.nlm.nih.gov/blast/db/).

Terminal 1. Using Blast to check the database
$ blastdbcmd -info -db /scratch/references/blastdb_update/blast-2021-09-01/db/nt Database: Nucleotide collection (nt) 72,899,005 sequences; 510,954,263,840 total bases Date: Aug 24, 2021 2:13 AM Longest sequence: 99,791,824 bases BLASTDB Version: 5 Volumes:   /scratch/references/blastdb_update/blast-2021-09-01/db/nt.00 ...

How to run Blast+ effectively on HPC

Blast+ is a single-node, multi-threaded application. A single instance of Blast+ cannot use more than one node.

SLURM resource request defaults

What resources should we request from SLURM for uncomplicated Blast+ jobs?

  • --nodes=1 because Blast+ cannot use more than one node

  • --ntasks=1because we are not using MPI parellelism

  • --cpus-per-task=Xwhere X is the number of threads Blast+ should use.

  • --memory= is variable depending on the size of the database you would like to use.

    • A reasonable default is 32GB which can be scaled up or down as required.

How to use job arrays for massive parallelism

Using a job array can really speed up your analysis by running many blast queries at once, rather than one at a time. Job arrays allow you take tasks that would have been done serially (i.e. one after the other) and do them in parallel (i.e. at the same time). A hypothetical set of 10 Blast searches which might have taken 10 hours serially might be completed in only 1 hour with parallelism.

Job arrays let you submit many near-identical jobs from one sbatch script. Slurm launches each of these jobs as an independent task, scheduled in parallel as resources free up. To use a job array, you must set the SLURM --array=<list> flag in your sbatch script. Each task gets a unique integer from your list, which becomes its $SLURM_ARRAY_TASK_ID. You can use this to assign each input file to a SLURM task, which will get run independently of all the other inputs. You can choose to use zero indexing, or not. The list can be formatted in two different ways: a comma-separated list of numbers, or a range of numbers specified using a dash "-". For example, --array=1,2,3,4 and --array=1-4 are functionally identical. These two formats can be combined, e.g. --array=1-2,5,8. An optional stride may be introduced when specifying a range using a colon ":". For example, --array=0-7:2" is equivalent to --array=0,2,4,6. For more details, see the section about job arrays on the Example Workflows page.

In the following example, we have 100 sequences to Blast, each with a unique number from 1-100 in the format chunk_[1-100].fa. This script will spawn 100 jobs which will each process one of the 100 .fa files.

Terminal 2. Example job array for massively parallel blast searches
#!/bin/bash -l #SBATCH --cpus-per-task=8 #SBATCH --mem=32G #SBATCH --time=02:00:00 #SBATCH --array=1-100 #Setting the array module load blast+/<version> blastp -query chunk_${SLURM_ARRAY_TASK_ID}.fa \ #each job gets a number from the array list above -db $DB \ -num_threads ${SLURM_CPUS_PER_TASK} \ -out result_${SLURM_ARRAY_TASK_ID}.tsv \ #using the same number again to write the output file with matching name -outfmt 6

Other tips for faster performance

The following tips can help to improve the efficiency of Blast+ queries:

  • Ensure queries do not contain exact replicas

  • Limit the number of search results. The default value for both -num_descriptions  and -num_alignments  (and the alternative -max_target_seqs ) is 500. These values should be reduced as much as possible to reduce both the time taken to perform the search and the resulting output file size.

  • Consider using the Blast+ option -outfmt 11 to produce ASN1 format. This option saves search results in a form that can be used to re-format the results using blast_formatter without the necessity of re-running the query (provided the relevant database is available). 

    • Depending on the exact type of search, the .asn1 file may be significantly larger (perhaps by a factor of 2-4) than the corresponding default text format. However, compressed .asn1  (produced using gzip for example) is usually smaller, making it a reasonable archival choice for large quantities of search data are to be archived.

External links