Frequently asked questions
Access
How to connect to front server?
Terminal
ssh stage01@front.migale.inrae.fr
Windows
For Windows, you need to use a terminal emulator tool such as MoBaXterm.
I got "Permission denied, please try again." when log in
You can reset it thanks to the dedicated form.
How to change my password?
If you lost your password, send a mail at help-migale@inra.fr. We will attribute you a new password and then you can change it easyly with the previous link.
How to copy a file from my computer to front server?
Terminal
scp myfile.txt stage01@front.migale.inrae.fr:/projet/tmp/
The password will be asked.
For copying a directory, add the -r option after scp
scp -r mydir stage01@front.migale.inrae.fr:/projet/tmp/
Windows
For Windows, you need to use a tool such as FileZilla.
Conda
How to show all conda environments?
stage01@front:~$ conda info --envs # conda environments: # base * /usr/local/genome/Anaconda2-5.1.0 BasicQC /usr/local/genome/Anaconda2-5.1.0/envs/BasicQC abyss-2.2.1 /usr/local/genome/Anaconda2-5.1.0/envs/abyss-2.2.1 admixture-1.3.0 /usr/local/genome/Anaconda2-5.1.0/envs/admixture-1.3.0 amos-3.1.0 /usr/local/genome/Anaconda2-5.1.0/envs/amos-3.1.0 ...
How to find a specific conda environment name?
stage01@front:~$ conda info --envs | grep bwa bwa-0.7.17 /usr/local/genome/Anaconda2-5.1.0/envs/bwa-0.7.17
How to use a tool with conda?
stage01@front:~$ conda activate bwa-0.7.17 (bwa-0.7.17) stage01@front:~$ bwa Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.17-r1188 Contact: Heng Li Usage: bwa command [options] Command: index index sequences in the FASTA format mem BWA-MEM algorithm fastmap identify super-maximal exact matches pemerge merge overlapping paired ends (EXPERIMENTAL) aln gapped/ungapped alignment samse generate alignment (single ended) sampe generate alignment (paired ended) bwasw BWA-SW for long queries shm manage indices in shared memory fa2pac convert FASTA to PAC format pac2bwt generate BWT from PAC pac2bwtgen alternative algorithm for generating BWT bwtupdate update .bwt to the new format bwt2sa generate SA from BWT and Occ Note: To use BWA, you need to first index the genome with `bwa index'. There are three alignment algorithms in BWA: `mem', `bwasw', and `aln/samse/sampe'. If you are not sure which to use, try `bwa mem' first. Please `man ./bwa.1' for the manual.
And to leave conda environment
stage01@front:~$ conda deactivate stage01@front:~$
How to use a conda environment within my snakemake workflow?
profile.d/conda.sh
. To do this, add these lines at the beginning of your Snakefile:
shell.executable("/bin/bash") shell.prefix("source /usr/local/genome/Anaconda3/etc/profile.d/conda.sh;")
In your snakemake rules, you can now use the conda environments with: conda activate my_env
. As a reminder, the list of available environments is accessible with:
conda info --envs
Moreover, if you launch snakemake inside a shell script, you need add the following line at the beginning of your script:
source /usr/local/genome/Anaconda3/etc/profile.d/conda.sh
This is a minimal example for a rule that launches fastqc
:
shell.executable("/bin/bash") shell.prefix("source /usr/local/genome/Anaconda3/etc/profile.d/conda.sh;") rule all: input: expand("qc/fastqc/{sample}_fastqc.html", sample = ["A", "B", "C"]) rule fastqc: input: "reads/{sample}.fastq.gz" output: html = "qc/fastqc/{sample}_fastqc.html", zip = "qc/fastqc/{sample}_fastqc.zip" params: "--quiet" log: "logs/fastqc/{sample}.log" threads: 1 shell: """ conda activate fastqc-0.11.9 fastqc {params} --threads {threads} --outdir qc/fastqc/ {input} > {log} conda deactivate """
More information here in the Snakemake FAQ.
If you obtain this error message:
CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run $ conda initCurrently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell See 'conda init --help' for more information and options. IMPORTANT: You may need to close and restart your shell after running 'conda init'.
check if you have a section called conda initialize in the file ~/.bashrc
and remove it.
How to use multiple threads?
-pe thread
to your submission command.
Example:
stage01@front:~$ qsub -pe thread 8 myscript.sh
What is the meaning of the Eqw state of my job?
Eqw
state indicates that your job is in error state. To obtain the reason why, use the qstat
command with your job-ID.
Example:
stage01@front:~$ qsub -cwd -V -N myblast -o /out/myblast.stdout -b y "blastn -db subject.fasta -query query.fasta -out /out/out.blast" Your job 1995192 ("myblast") has been submitted stage01@front:~$ qstat job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 1995192 0.00000 myblast stage01 Eqw 10/04/2019 17:11:46 1 stage01@front:~$ qstat -j 1995192 ... error reason 1: 10/16/2019 14:35:39 [46342:142027]: error: can't open output file "/out/myblast.stdout": No such file or directory ...
Why my job never starts?
stage01@front:~$ qstat job-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 2039350 0.60500 myblast stage01 qw 10/16/2019 14:44:34 100
In some cases, jobs are never started and will never be started. In this example, the number of reserved threads is higher than what the nodes offer. As a result, SGE waits until it has a 100-slot node at its disposal to start your job. To check if the waiting is normal, use the qstat command with the job number to check that there is no warning as in this example. You will find here the detail about our resources.
stage01@front:~$ qstat -j 2039350 ... cannot run in PE "thread" because it only offers 0 slots ...
How do I know when my jobs are finished?
qsub
):
-m ea
Will send email when job ends or aborts-M <emailaddress>
Email address to send email to
If you keep working on front, you can use the command watch
. Watch is used to run any arbitrary command at regular intervals and displays the output of the command on the terminal window. It is useful when you have to execute a command repeatedly and watch the command output change over time. In this case, the output of qstat will be empty.
stage01@front:~$ watch -n 5 "qstat" # Will run qstat every 5 seconds
My quota does not decrease despite the deletion of datasets!
Why can't I build a workflow from my history?
There is an error message at Galaxy's address!
How to archive and compress files?
Create tar.gz archive file
stage01@front:~$ tar cvzf archive_name.tar.gz folder_to_archive
Untar and uncompress tar.gz archive file
stage01@front:~$ tar xvfz archive_name.tar.gz
tar usage and options
- c create a archive file
- x extract a archive file
- v show the progress of archive file
- f filename of archive file
- t viewing content of archive file
- j filter archive through bzip2
- z filter archive through gzip
How to Fix the ‘No Space Left on Device’ Error?
get_quota
stage01@front:~$ get_quota Information for stage01 at mar. 01 févr. 2022 09:45:22 CET Home : 110.8MB / 1GB (11%) Save : 0B / 50GB (0%) Work : 8KB / 150GB (0%)
And to see details by folder you can use the du
command
stage01@front:~$ du -sh ~/myfolder/* 1,7G /home/stage01/myfolder/checkm-genome-1.0.18 8,0K /home/stage01/myfolder/checkm_test 8,0K /home/stage01/myfolder/checkm_test_results 4,0K /home/stage01/myfolder/dir1 ...
I can't find the path to perl
perl
was /usr/local/public/perl-5-24.0/bin/perl
Now, the path to perl is /usr/bin/perl
stage01@front:~$ which perl /usr/bin/perl
Default Python has changed
stage01@front:~$ python --version Python 3.8.10
Python 2 is no more maintained since end of 2019. We strongly recommand you to use Python3 if you begin a new project, or to upgrade your code to Python3. You can find details on what changed between the two versions.