Cori is still having a lot of outages at NERSC (most of them scheduled), but Tom reported on some promising news from a short window when demand was low: the DC1 PhoSim Deep image simulation jobs were able to be executed pretty efficiently, which bodes well. Tony reported that we are now over halfway through the `processEimage` phase of the Level 2 processing - coadd creation and image differencing will follow this. We then turned to our DESC Note setup, making a Note from scratch live on the call, so that people see how to use `start_paper`.
We’ll go ahead with our plan following no dissent on Slack! The Twinkles team leader position remains open.
News from NERSC? Several shutdowns over the last week or two. Recovery capability from shutdown seems to be under development. Part of this is on us: we rely on some cron jobs, which may go down with the hardware. Other problems include: lack of login, disappearance of file system. Tom thinks that we are helping them somewhat in identifying necessary recovery steps, which is cool.
Some success on Monday before outage: DESC CI managed to get hold of 170 cores, to make 1700 simultaneous DC1 PhoSim jobs. Unusual situation, in window right before scheduled shutdown… but our ability to scale up PhoSim was demonstrated.
Tony reported that 13000 out of 21000 processEimage jobs are now complete, which is great. He will keep pushing on these; the next step will be coadd creation and then image differencing. Simon suggested doing some tests on a small multi-filter subset while we wait, Tony will produce one.
Phil has been swamped, and still has not set up branches with `start_paper` notes in them. We made the Pserv note live, so that everyone could see how to make their own.
The Twinkles Notes folder contains a README, to be kept up to date with the locations of the Notes and their various states.
The Epic on Twinkles 1 LSST DESC Notes (#284) contains our list of Notes. Find your note in the list.
Issue yourself with the making of the note (search issues first to avoid dup issues).
In the Twinkles repo (or wherever your note should live) make a branch like: issue/issuenumber/note_name
Go to Twinkles/docs (or wherever your note should live) -- the `start_paper` cookiecutter will put a folder in this directory.
Follow directions at https://github.com/LSSTDESC/start_paper
The cookiecutter command leads to a set of questions you should answer. Answer them as best you can.
Note: serial number is not set initially. The pub board will give you one at some point. Just accept 0000.
You can choose the format, but it will produce the supporting material for all the possible formats. The possible formats currently are: tex, apj, mnras, prl, prd, ipynb, md, rst.
Edit the `authors.csv` file, following the directions in the header.
NB. Contributions are not timecard, but instead a summary of your most important contributions.
Open editor or notebook (depending on your chosen format), start editing the appropriate `main.*` file.
Note authors are not currently read from authors.csv in ipynb, rst, md formats, so author lists need to be maintained by hand… This is a start_paper bug.
You can leave the other format files lying around, in case you decide to switch formats.
When you are ready, `git add` all the files you just created. Don’t forget to `git add .*` as well, to catch the .files.
`git commit` and `push` to a new remote branch of the same name.
Open the pull request, so conversation can commence.
Don’t assign reviewers until the Note is “finished” and ready to be merged to `master`.
Don’t forget to refer to the issue, so that the PR and issue are correctly hyperlinked together. You might also refer to #284 (the notes epic)