[WAM-IPE] Realtime wam-ipe runs on theia: archive

Mon Jul 24 22:45:34 UTC 2017

In /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe

1. The following are already saved:

   - plasmaXX:
   - ut_rec
   - IPE.inp
   - stupXX

Note: Since stup* files are from previous run, they are already saved in
the previous (6 hours earlier) run dir. The soft link in the current run
dir are for the NEXT run. I will remove these soft link (and change my run
scripts accordingly), because this can be confusing.

2. The following will be saved in the future:

   - FLIP_ERR
   - logfile_input_params.log

3. The following are saved to mass storage (HPSS): they are called
'dayfile' which contain both output:

   - standard output
   - standard error output

I have saved the entire rundir for two runs in:

/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071606gdasfcst1
/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071612gdasfcst1

and the dayfile:

/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/
wam2017071606gdasfcst1_0.dayfile
/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/
wam2017071612gdasfcst1_0.dayfile

and the figs of NmF2 in

/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/fig/

Let me know if I missed anything.

(I will compare NmF2 in the forecast and in the initial conditions used for
restart.)

Houjun Wang

On Thu, Jul 20, 2017 at 1:23 PM, Naomi Maruyama - NOAA Affiliate <
naomi.maruyama at noaa.gov> wrote:

> Houjun and Adam,
>
> here is a list of files I would like to suggest to be saved under your
> archive(/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe)
>
>
>    - plasmaXX
>    - ut_rec
>    - IPE.inp
>    - stupXX
>    - FLIP_ERR
>    - logfile_input_params.log
>    - standard output
>    - standard error output
>
>
>
> Theses files would be helpful if you want me to look at your runs and
> verify the results.
>
> Thanks.
> naomi
>
>
>
> On Wed, Jul 12, 2017 at 2:30 PM, Adam Kubaryk - NOAA Affiliate <
> adam.kubaryk at noaa.gov> wrote:
>
>> Hi Naomi and Houjun,
>>
>> I'll chime in quickly on the parallel script workflow.
>>
>> That para_config file is used to submit a job that:
>>
>> runs /scratch3/NCEPDEV/swpc/svn/gfs/branches/gfs_workflow.v2.0.0/
>> para/jobs/fcst.sh
>> fcst.sh immediately calls the para_config before running anything else
>> para_config then calls /scratch3/NCEPDEV/swpc/svn/gfs
>> /branches/gfs_workflow.v2.0.0/para/ush/reconcile.sh
>> after reconcile.sh, para_config finishes as well, and then fcst.sh runs
>> fcst.sh calls /scratch3/NCEPDEV/swpc/svn/gfs
>> /branches/global_shared.v14.0.0/scripts/exglobal_fcst_nems.s
>> h.ecf-wam-ipe_wdas_iau2
>>
>> A lot of what's relevant to IPE itself (copying in or linking files to
>> the RUNDIR, setting up the namelists, etc.) is going to occur in that final
>> exglobal script.
>>
>> Let me know if there's anything else I can clarify.
>>
>> Best,
>> Adam
>>
>> On Wed, Jul 12, 2017 at 4:22 PM, Houjun Wang - NOAA Affiliate <
>> houjun.wang at noaa.gov> wrote:
>>
>>> 1. Yes, we have hourly output from ipe and since this WDAS-IAU cycled
>>> runs, each run only have 12 hour duration (and since ipe doesn't output the
>>> last time step - George know this well and may have a fix for it now - it
>>> does't have 15 UT for example).
>>>
>>> So this is not an issue.
>>>
>>> 2. The source code is in
>>>
>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU
>>>
>>> see especially the mods in
>>>
>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU/IPELI
>>> B/src/plasma/io_plasma_bin.f90
>>>
>>> for initializing ipe from ipe output from previous cycle in WDAS-IAU.
>>>
>>> I added another option "sw_record_number=2" for this, see IPE.inp for
>>> more details and other changes. I also reset record number to zero for each
>>> run in WDAS-IAU cycles to make it more practical for this purpose.
>>>
>>> 3. Now, since I am using EMC para workflow, there is not just one script.
>>>
>>> The config file I am using is:
>>>
>>> /scratch3/NCEPDEV/swpc/save/swpc.spacepara/para/para_config_
>>> wam_ipe_wdas_iau
>>>
>>> which defines all the runs.
>>>
>>> Kate wrote some notes on using EMC para scripts/workflow for wam/wdas:
>>>
>>> https://docs.google.com/a/noaa.gov/document/d/1OmlGZc0WQy8kd
>>> jdPWqHLQZUVn47fXqTn41rbgyrCguc/edit?usp=sharing
>>>
>>> l'll see if I can answer if you have questions.
>>>
>>> Houjun Wang
>>>
>>> On Wed, Jul 12, 2017 at 1:24 PM, Naomi Maruyama - NOAA Affiliate <
>>> naomi.maruyama at noaa.gov> wrote:
>>>
>>>> Houjun,
>>>>
>>>> Thank you very much for your kindly sharing your output files.
>>>>
>>>> I have a feeling that your IPE output files were generated from only
>>>> 3UT to 14UT only for each day, after looking at some of your IPE ARCHIVE
>>>> directory.
>>>>
>>>> In order to understand this potential issue in further detail, I would
>>>> need to understand how your restart of IPE has been set up in your script
>>>> and source code. Would you mind sharing your script and source code with
>>>> which you have generated these output files?
>>>>
>>>> Thanks.
>>>> naomi
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Jul 11, 2017 at 3:49 PM, Houjun Wang - NOAA Affiliate <
>>>> houjun.wang at noaa.gov> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> Some info on realtime wam-ipe runs on theia:
>>>>>
>>>>> 1) RUNDIR (this is temporary dir and it could be gone anytime)
>>>>>
>>>>> /scratch4/NCEPDEV/stmp4/swpc.spacepara/wam2017070506gdasfcst1
>>>>>
>>>>> 2) ARCHIVE (not all files are saved for ipe; what else should be
>>>>> saved?)
>>>>>
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe
>>>>>
>>>>> 3) HPSS ARCHIVE (George can get ICs for WAM here):
>>>>>
>>>>> /5year/NCEPDEV/swpc-para/swpc.spacepara/THEIA/prwam_ipe
>>>>>
>>>>> Let me know if you have any questions or suggestions.
>>>>>
>>>>> Best,
>>>>>
>>>>> Houjun Wang
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Naomi Maruyama
>>>> CIRES, Univ. of Colorado Boulder
>>>> NOAA Space Weather Prediction Center
>>>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>>>> email: naomi.maruyama at noaa.gov
>>>>
>>>>
>>>
>>> _______________________________________________
>>> WAM-IPE mailing list
>>> WAM-IPE at lstsrv.ncep.noaa.gov
>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/wam-ipe
>>>
>>>
>>
>
>
> --
>
> Naomi Maruyama
> CIRES, Univ. of Colorado Boulder
> NOAA Space Weather Prediction Center
> phone: +1 303.497.4857 <%28303%29%20497-4857>
> email: naomi.maruyama at noaa.gov
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/wam-ipe/attachments/20170724/b2409a10/attachment-0001.html 