Senior DevOps/Data Services Engineer - Rubin Observatory

The Vera C. Rubin Observatory (LSST) Data Management team seeks an individual with demonstrated experience in developing and managing scalable services and infrastructure. This person will be responsible for inter alia developing services for the Rubin Science Platform. This is an opportunity to do current best-practices work in support of one of the defining scientific missions in modern-day Astronomy.

The Legacy Survey of Space and Time will be carried out with our large-aperture, wide-field, ground-based telescope that will survey half the sky every few nights in six optical bands. It will explore a wide range of astrophysical questions, ranging from discovering “killer” asteroids, to examining the nature of dark energy. The project includes an 8-meter class wide field telescope, a 3.2 gigapixel camera with 2-second readout, and a state-of-the-art petascale data management system to process, archive, and distribute the 15 TB of data produced every night. Once completed, the LSST will be the largest and most modern optical survey ever made.

The LSST Data Management (DM) team is a distributed team of ~100 individuals in various partner institutions. It is building a data processing system spanning two continents, new state-of-the-art image processing algorithms, petascale computing clusters with tens of thousands of cores, large distributed databases, and next-generation Rubin Science Platform (RSP), among others. All LSST DM code is published on GitHub and released under a major open-source license; our lingua franca is Python 3. This position has its home in the Science Quality and Reliability Engineering (SQuaRE) team. SQuaRE is a high-performing DevOps team that has developed and deployed many services supporting software development and observatory operations, including much of the Rubin Science Platform. As well as making all our software open source (http://github.com/lsst-sqre/), SQuaRE also frequently upstreams to open-source projects that we consume.

The position reports to the SQuaRE Technical Manager (Rubin Data Services Lead in Operations). This full-time position is based in Tucson, Arizona. Exceptional candidates may be considered for a mostly remote role.

Essential Functions: 

• Designs, implements (predominantly in Python) and supports infrastructure and services running on the Rubin Science Platform and other Rubin Data Services platforms
• Ensures (by architecture and in practice) that services are scalable, robust, maintainable and secure
• Contributes to the SQuaRE common toolchain (GitHub Actions, configuration management, service deployment infrastructure, etc)
• Writes lucid documentation that effectively captures architectural and implementation choices
• Shares responsibility with other team-members in doing self-directed work that nevertheless fits in a coherent overall system vision reached through shared understanding and team consensus
• Productively engages with the SQuaRE team lead to keep her well informed of technical decisions, task progress and discuss architectural implications of ongoing work
• Participates in the collective engineering activities of the SQuaRE team, including performing code reviews, acting as a troubleshooting buddy, contributing to refactoring sprints, engaging in (vigorous but good-natured) technical debates, joining in the team daily stand-ups and weekly co-working session
• All AURA employees are responsible for the proper management and control of all AURA property within their work area, whether assigned to them or someone else. This responsibility includes reporting any known missing, stolen, or non-working property in their work area to their supervisor or the AURA Property Officer.
• Responsible for ensuring compliance with government award terms and conditions assigned to this position or subordinates of this position as defined in the Vela Award Management system.

Required Education/Experience/Skills/Abilities:

SQuaRE is a generalist team with the capacity to adjust to a highly skilled individual that may not be an exact fit to a single set of requirements, but that possesses deep equivalent experience. We encourage you not to self-censor and to apply if you feel you fall in this category, addressing how you feel you will be a good match to the needs that give rise to these requirements in your cover letter. 

• Hands-on experience architecting, implementing and supporting code in a service architecture (typically 10+ years of experience in a professional capacity at a technically challenging organization or company)
• Fluency in Python 3, including advanced topics such as data structures, pydantic, async, mypy and REST frameworks such as FastAPI
• Excellent software engineering skills such as writing clear, well-encapsulated code, with readable documentation and unit and CI tests
• Experience supporting production services on cloud platforms such as Google Cloud or AWS
• Familiarity with Docker, or similar containerization technologies
• Extensive familiarity with git and distributed development workflows based on online code management systems (preferably GitHub)
• Excellent written communication skills in English, including the ability to lucidly explain complex technical topics to colleagues that may not share the same domain expertise
• Willingness to travel occasionally and represent our work
  Comfortable collaborating with individuals/teams remotely with chat (e.g. Slack) and teleconferencing
• Being able to work in a self-directed manner in pursuit of the team’s technical goals

Preferred Education/Experience/Skills/Abilities: 

• Successful team members typically have these additional qualifications
• Master’s degree or higher in computer science / information technology / data science, OR demonstrable equivalent experience in the field in terms of being able to think analytically and synthetically about the subject, express abstract concepts, ability to reason systematically when troubleshooting etc.
• Familiarity with Kubernetes
• Comfortable with GitOps, including GitOps tooling such as ArgoCD
• Familiarity with Github services, particularly Github Actions
• Fluency in another language common in full stack engineering (e.g. go, JavaScript)
• Desire to contribute to open source software
• An interest in astronomy

Physical Demands: 

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

• The work is mostly carried out in an office environment.  Long periods of sitting and computer usage may be required.
• Some travel may be necessary in or outside the contiguous United States.

How to Apply: 

The position will remain open until it is filled. To be considered in the first round of reviews, please submit your application by November 21, 2023.

• Please submit a cover letter and a CV or resume, and links to any open-source contributions that you would like to draw to our attention.
• PDF files preferred please name any attachments with the following format: Lastname Docname. 
• Application documents that are not uploaded as part of the application may be sent to employment@aura-astronomy.org. 
• Please enter 3 professional references in your application (references will only be contacted later in the recruitment process). At least one should have acted in a leadership capacity for a team you were a member of.

Veterans, disabled individuals or wounded warriors needing assistance with the employment process should request assistance at employment@aura-astronomy.org 

The Association of Universities for Research in Astronomy (AURA) is a consortium of 49 US institutions and 3 international affiliates that operates world-class astronomical observatories for the National Science Foundation and NASA. AURA’s role is to establish, nurture, and promote public observatories and facilities that advance innovative astronomical research. In addition, AURA is deeply committed to public and educational outreach, and to diversity throughout the astronomical and scientific workforce. AURA carries out its role through its astronomical facilities. 

AURA offers an excellent benefits package including paid time off and retirement plan contributions, competitive salary commensurate with experience, and a very attractive work environment. Details on benefits can be found at https://hr.aura-astronomy.org/employee-benefits-guide/ 

Consistent with the actions of the Biden Administration, as of 12 May 2023, AURA will no longer require COVID-19 vaccination of its employees in the U.S. AURA employees in the U.S. may be subject to COVID-19 vaccination requirements from other entities, such as host institutions or partner affiliations. 

As a recipient of U.S. Government funding, AURA is considered a government contractor and is subject to Equal Employment Opportunity and Affirmative Action regulations. As an Equal Opportunity and Affirmative Action Employer, AURA does not discriminate based on race, sex, color, age, religion, national origin, sexual orientation, gender identity/gender identity expression, lawful political affiliations, veteran status, disability, and/or any other legally protected status under applicable federal, state, and local equal opportunity laws. The statements below as well as the requests for self-identification are required pursuant to these regulations. We encourage your participation in meeting these federal reporting requirements which are included for protection and to assist us in our recordkeeping and reporting. Your responses are kept strictly confidential.