Before determining which option is best for you, it is helpful to spend some time planning your project. We recommend to talk to colleagues, peers, and your lab. We also recommend talking to us regarding project planning that involves genomic or bioinformatics data. This involves thinking about your research question, how it integrates with wet lab experiments, data types to answer your question, planned amount of data to be analyzed and how, data policies associated with different grants, and pre and post-project genomics data management. Please contact us if you need help getting started!
How computing options are organized at UW-Madison:
At UW-Madison, we have “departmental” and “shared” computing infrastructure. Departmental services are available to members of given departments, and are sometimes associated with membership or a cost. Shared Computing Infrastructure on campus refers to resources that are available for no-fee to the user, and available to any UW-Madison researcher.
There is no need to get a new computer to access any of these resources, as you will connect to “their machines” using the Terminal application on your Mac, Windows or Linux machine; but these resources are distributed across many departments, centers and offices on campus.
In Bacteriology
- Lab hosted servers: Some labs in the department of Bacteriology manage their own laboratory server. Your PI would know the information on how to log in to those. Not all labs have this. Often Linux based most of the time.
- Center for High-Throughput Computing (free, available to all, but no sensitive data): Commonly referred as “CHTC”, this is a computing option available to all members of UW-Madison, free of charge for research-purposes. CHTC has both High Throughput and High-Performance computing options. The Bacteriology Bioinformatics Research Support Service works closely with its members to onboard, teach, and implement how to use CHTC systems to run bioinformatics workflows. However, this is not and option for you if the data you will be using is sensitive or restricted (e.g. HIPAA protected). If you are unsure, speak to us. This is the most popular option among Bacteriology labs due to the cost, flexibility and use cases. If you would like to get started using CHTC, please visit the CHTC website to Request an Account. We provide end-to-end mentoring on how to initiate a project, learn how to submit jobs, and work together to develop bioinformatics workflows to be run on CHTC.
- “Centers”: Centers overlap departments, and different faculty members can be part of different centers. Many Bacteriology Faculty are members of the Great Lakes Bioenergy Research Center. The GLBRC has a compute server and staff dedicated to help with bioinformatics. We can assist researchers wanting to learn how to use HTCondor to run bioinformatics jobs on the GLBRC servers, however we direct questions related to the GLBRC staff.
- UW-Madison Cloud Campus Infrastructure: UW-Madison DoIT Research Cyberinfrastructure manages cloud servers through vendor deals (Amazon, Google, Microsoft) with UW-Madison, and are available to any lab/department. With this option, your lab would pays for a server that you can manage, with support from DOIT. This is a pay-per-use option. They host office hours (see Calendar tab) each week if you are curious about this option. We have not yet provided CCI support within the department at the moment but let us know if this is something you are interested in learning more about.
Outside Bacteriology
We do not mentor researchers directly on how to use these services. Please refer to their own pages for more information.
- Certains labs in the same building as the Department of Bacteriology, are under the UW School of Medicine and Public Health. The UW SMPH has these two computing resources:
- Platform X: Research computing environment that is scalable to researchers needs to receive and analyze data; HIPAA compliant; hosted in cloud by UW SMPH
- Platform R: Research Linux-based high-performance computing environment, with GPUs, many CPUs, and temporary fast ceph storage, HIPAA compliant; hosted locally by UW SMPH
- Social Science Computing Cooperative (SSCC) (restricted, sensitive data): The Department of Bacteriology is not a member of the SSCC, so we do not have access to these servers and statistical consulting services. If you need their services, visit https://www.sscc.wisc.edu/about-the-ssc/join-the-sscc/ to learn about Individual Memberships in the Cooperative.
- Departmental servers: Various departments on campus have compute servers. For example, some researchers in the Department of Faculty are affiliated with: the Department of Chemistry department has a High Performance Computing (HPC) cluster, and the Department of Biochemistry has a High Throughput Computing (HTC) computing cluster.
Data Storage Options
Please use this tool to help you decide where you should store you data throughout a project.
Every faculty in the Department of Bacteriology have access to a “ResearchDrive” account, which stores 25TB of data, with more space available for an annual fee. This is a good option to store bioinformatics/research related data as the file sizes can be quite large.
The ResearchDrive account can be connected to compute resources, such as CHTC, for a streamlined workflow.
We have support-staff levels account for ResearchDrive and can teach how to use use ResearchDrive, how to use it in jobs on CHTC, etc. However, for permissions issues we will refer you to the ResearchDrive team.
Moving Data
Omics & bioinformatics data can be several gigabytes (GBs) and even terabytes (TBs) in size. The best way to move this type of data is by using Globus. Moving large amounts of data over wifi can be not secure and can fail (if the internet goes down). Globus is a UW-Madison supported service that enables people to transfer large amount of data between locations (for example, a CHTC folder, a ResearchDrive, a personal computer, a remote server of your choice) in a secure & fast way.
We can teach you how to set up Globus to transfer files between endpoints.