[WAM-IPE] Realtime wam-ipe runs on theia: archive

Thu Jul 27 17:53:44 UTC 2017

I am working with the latest trunk version now, and from now on. Any
changes you want to add to the trunk, I can get from there directly.

Houjun Wang

On Thu, Jul 27, 2017 at 11:19 AM, Naomi Maruyama - NOAA Affiliate <
naomi.maruyama at noaa.gov> wrote:

> Rashid and Houjun,
>
> yes, I can suggest how to change the IPE code in order to stop the run
> immediately when a flux tube fails first.
>
> Houjun:
> could you please commit your IPE branch to our IPE so that I can easily
> search for the differences between your IPE version vs. the IPE master in
> the IPE repository on github (please also the instruction on how to work
> with WAM-IPE branch:https://docs.google.com/a/noaa.gov/document/d/
> 1iGSijW5c-y4YW29bXhZhO0SzXUz3Ywjz0e1r-xHI7hk/edit?usp=sharing). I am
> going to send you an invitation as a collaborator from github.
>
> Thanks.
> naomi
>
> On Tue, Jul 25, 2017 at 5:32 PM, Rashid Akmaev - NOAA Federal <
> rashid.akmaev at noaa.gov> wrote:
>
>> Is it possible to catch when a tube first fails? Does the restart from
>> MSIS Houjun uses make them fail? Plotting composition after the fact
>> probably may not help much as failed tubes will most likely stay that way
>> no matter what the composition, right?
>>
>> Rashid Akmaev
>>
>> On Jul 25, 2017, at 4:25 PM, Naomi Maruyama - NOAA Affiliate <
>> naomi.maruyama at noaa.gov> wrote:
>>
>> Hi Houjun,
>>
>> Thanks for your showing some updates at our meeting. Your results look
>> very encouraging. But could you please also plot the WAM fields from your
>> runs in order to confirm that the WAM fields that are used in IPE are
>> reasonable?
>>
>> In your plotting script, set the VarType to 4 (instead of 7 for NmF2) if
>> you want to plot WAM neutral atomic Oxygen, for example. Also you could
>> change the altitude of your plot by changing the line, "setenv alt" inside
>> the visualization script.
>>
>> Thanks.
>> naomi
>>
>>
>>
>>
>>
>>
>> On Tue, Jul 25, 2017 at 2:29 PM, Naomi Maruyama - NOAA Affiliate <
>> naomi.maruyama at noaa.gov> wrote:
>>
>>> Thanks.
>>>
>>> I see that failed flux tubes in your plots and FLIP_ERR in your runs.
>>> But I cannot suggest anything to solve the problem until I can make some
>>> test runs.
>>>
>>> In order to solve this issue, I would need to reproduce your run by
>>> myself, and try to debug the run. Could you please suggest how I can do
>>> that?
>>>
>>> Thanks.
>>> naomi
>>>
>>> On Tue, Jul 25, 2017 at 10:30 AM, Houjun Wang - NOAA Affiliate <
>>> houjun.wang at noaa.gov> wrote:
>>>
>>>> yes
>>>>
>>>> Houjun Wang
>>>>
>>>> On Tue, Jul 25, 2017 at 10:25 AM, Naomi Maruyama - NOAA Affiliate <
>>>> naomi.maruyama at noaa.gov> wrote:
>>>>
>>>>> Houjun
>>>>>
>>>>> Thank you very much for archiving the files. I am going to study them.
>>>>>
>>>>>
>>>>>
>>>>> >>I have saved the entire rundir for two runs in:
>>>>>
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>> 7071606gdasfcst1
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>> 7071612gdasfcst1
>>>>>
>>>>> you mentioned at the previous meeting that IPE is slowing down with
>>>>> failed flux tubes. Are these two runs have the problem?
>>>>>
>>>>> Thanks.
>>>>> naomi
>>>>>
>>>>> On Mon, Jul 24, 2017 at 4:45 PM, Houjun Wang - NOAA Affiliate <
>>>>> houjun.wang at noaa.gov> wrote:
>>>>>
>>>>>> In /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe
>>>>>>
>>>>>> 1. The following are already saved:
>>>>>>
>>>>>>    - plasmaXX:
>>>>>>    - ut_rec
>>>>>>    - IPE.inp
>>>>>>    - stupXX
>>>>>>
>>>>>> Note: Since stup* files are from previous run, they are already saved
>>>>>> in the previous (6 hours earlier) run dir. The soft link in the current run
>>>>>> dir are for the NEXT run. I will remove these soft link (and change
>>>>>> my run scripts accordingly), because this can be confusing.
>>>>>>
>>>>>> 2. The following will be saved in the future:
>>>>>>
>>>>>>    - FLIP_ERR
>>>>>>    - logfile_input_params.log
>>>>>>
>>>>>>
>>>>>> 3. The following are saved to mass storage (HPSS): they are called
>>>>>> 'dayfile' which contain both output:
>>>>>>
>>>>>>    - standard output
>>>>>>    - standard error output
>>>>>>
>>>>>>
>>>>>> I have saved the entire rundir for two runs in:
>>>>>>
>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>>> 7071606gdasfcst1
>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>>> 7071612gdasfcst1
>>>>>>
>>>>>> and the dayfile:
>>>>>>
>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>>> 7071606gdasfcst1_0.dayfile
>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam201
>>>>>> 7071612gdasfcst1_0.dayfile
>>>>>>
>>>>>> and the figs of NmF2 in
>>>>>>
>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/fig/
>>>>>>
>>>>>> Let me know if I missed anything.
>>>>>>
>>>>>> (I will compare NmF2 in the forecast and in the initial conditions
>>>>>> used for restart.)
>>>>>>
>>>>>>
>>>>>> Houjun Wang
>>>>>>
>>>>>> On Thu, Jul 20, 2017 at 1:23 PM, Naomi Maruyama - NOAA Affiliate <
>>>>>> naomi.maruyama at noaa.gov> wrote:
>>>>>>
>>>>>>> Houjun and Adam,
>>>>>>>
>>>>>>> here is a list of files I would like to suggest to be saved under
>>>>>>> your archive(/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archiv
>>>>>>> e/ipe)
>>>>>>>
>>>>>>>
>>>>>>>    - plasmaXX
>>>>>>>    - ut_rec
>>>>>>>    - IPE.inp
>>>>>>>    - stupXX
>>>>>>>    - FLIP_ERR
>>>>>>>    - logfile_input_params.log
>>>>>>>    - standard output
>>>>>>>    - standard error output
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Theses files would be helpful if you want me to look at your runs
>>>>>>> and verify the results.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> naomi
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 12, 2017 at 2:30 PM, Adam Kubaryk - NOAA Affiliate <
>>>>>>> adam.kubaryk at noaa.gov> wrote:
>>>>>>>
>>>>>>>> Hi Naomi and Houjun,
>>>>>>>>
>>>>>>>> I'll chime in quickly on the parallel script workflow.
>>>>>>>>
>>>>>>>> That para_config file is used to submit a job that:
>>>>>>>>
>>>>>>>> runs /scratch3/NCEPDEV/swpc/svn/gfs/branches/gfs_workflow.v2.0.0/
>>>>>>>> para/jobs/fcst.sh
>>>>>>>> fcst.sh immediately calls the para_config before running anything
>>>>>>>> else
>>>>>>>> para_config then calls /scratch3/NCEPDEV/swpc/svn/gfs
>>>>>>>> /branches/gfs_workflow.v2.0.0/para/ush/reconcile.sh
>>>>>>>> after reconcile.sh, para_config finishes as well, and then fcst.sh
>>>>>>>> runs
>>>>>>>> fcst.sh calls /scratch3/NCEPDEV/swpc/svn/gfs
>>>>>>>> /branches/global_shared.v14.0.0/scripts/exglobal_fcst_nems.s
>>>>>>>> h.ecf-wam-ipe_wdas_iau2
>>>>>>>>
>>>>>>>> A lot of what's relevant to IPE itself (copying in or linking files
>>>>>>>> to the RUNDIR, setting up the namelists, etc.) is going to occur in that
>>>>>>>> final exglobal script.
>>>>>>>>
>>>>>>>> Let me know if there's anything else I can clarify.
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Adam
>>>>>>>>
>>>>>>>> On Wed, Jul 12, 2017 at 4:22 PM, Houjun Wang - NOAA Affiliate <
>>>>>>>> houjun.wang at noaa.gov> wrote:
>>>>>>>>
>>>>>>>>> 1. Yes, we have hourly output from ipe and since this WDAS-IAU
>>>>>>>>> cycled runs, each run only have 12 hour duration (and since ipe doesn't
>>>>>>>>> output the last time step - George know this well and may have a fix for it
>>>>>>>>> now - it does't have 15 UT for example).
>>>>>>>>>
>>>>>>>>> So this is not an issue.
>>>>>>>>>
>>>>>>>>> 2. The source code is in
>>>>>>>>>
>>>>>>>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU
>>>>>>>>>
>>>>>>>>> see especially the mods in
>>>>>>>>>
>>>>>>>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU/IPELI
>>>>>>>>> B/src/plasma/io_plasma_bin.f90
>>>>>>>>>
>>>>>>>>> for initializing ipe from ipe output from previous cycle in
>>>>>>>>> WDAS-IAU.
>>>>>>>>>
>>>>>>>>> I added another option "sw_record_number=2" for this, see IPE.inp
>>>>>>>>> for more details and other changes. I also reset record number to zero for
>>>>>>>>> each run in WDAS-IAU cycles to make it more practical for this purpose.
>>>>>>>>>
>>>>>>>>> 3. Now, since I am using EMC para workflow, there is not just one
>>>>>>>>> script.
>>>>>>>>>
>>>>>>>>> The config file I am using is:
>>>>>>>>>
>>>>>>>>> /scratch3/NCEPDEV/swpc/save/swpc.spacepara/para/para_config_
>>>>>>>>> wam_ipe_wdas_iau
>>>>>>>>>
>>>>>>>>> which defines all the runs.
>>>>>>>>>
>>>>>>>>> Kate wrote some notes on using EMC para scripts/workflow for
>>>>>>>>> wam/wdas:
>>>>>>>>>
>>>>>>>>> https://docs.google.com/a/noaa.gov/document/d/1OmlGZc0WQy8kd
>>>>>>>>> jdPWqHLQZUVn47fXqTn41rbgyrCguc/edit?usp=sharing
>>>>>>>>>
>>>>>>>>> l'll see if I can answer if you have questions.
>>>>>>>>>
>>>>>>>>> Houjun Wang
>>>>>>>>>
>>>>>>>>> On Wed, Jul 12, 2017 at 1:24 PM, Naomi Maruyama - NOAA Affiliate <
>>>>>>>>> naomi.maruyama at noaa.gov> wrote:
>>>>>>>>>
>>>>>>>>>> Houjun,
>>>>>>>>>>
>>>>>>>>>> Thank you very much for your kindly sharing your output files.
>>>>>>>>>>
>>>>>>>>>> I have a feeling that your IPE output files were generated from
>>>>>>>>>> only 3UT to 14UT only for each day, after looking at some of your IPE
>>>>>>>>>> ARCHIVE directory.
>>>>>>>>>>
>>>>>>>>>> In order to understand this potential issue in further detail, I
>>>>>>>>>> would need to understand how your restart of IPE has been set up in your
>>>>>>>>>> script and source code. Would you mind sharing your script and source code
>>>>>>>>>> with which you have generated these output files?
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>> naomi
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 11, 2017 at 3:49 PM, Houjun Wang - NOAA Affiliate <
>>>>>>>>>> houjun.wang at noaa.gov> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> Some info on realtime wam-ipe runs on theia:
>>>>>>>>>>>
>>>>>>>>>>> 1) RUNDIR (this is temporary dir and it could be gone anytime)
>>>>>>>>>>>
>>>>>>>>>>> /scratch4/NCEPDEV/stmp4/swpc.spacepara/wam2017070506gdasfcst1
>>>>>>>>>>>
>>>>>>>>>>> 2) ARCHIVE (not all files are saved for ipe; what else should be
>>>>>>>>>>> saved?)
>>>>>>>>>>>
>>>>>>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe
>>>>>>>>>>>
>>>>>>>>>>> 3) HPSS ARCHIVE (George can get ICs for WAM here):
>>>>>>>>>>>
>>>>>>>>>>> /5year/NCEPDEV/swpc-para/swpc.spacepara/THEIA/prwam_ipe
>>>>>>>>>>>
>>>>>>>>>>> Let me know if you have any questions or suggestions.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>>
>>>>>>>>>>> Houjun Wang
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Naomi Maruyama
>>>>>>>>>> CIRES, Univ. of Colorado Boulder
>>>>>>>>>> NOAA Space Weather Prediction Center
>>>>>>>>>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>>>>>>>>>> email: naomi.maruyama at noaa.gov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> WAM-IPE mailing list
>>>>>>>>> WAM-IPE at lstsrv.ncep.noaa.gov
>>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/wam-ipe
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Naomi Maruyama
>>>>>>> CIRES, Univ. of Colorado Boulder
>>>>>>> NOAA Space Weather Prediction Center
>>>>>>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>>>>>>> email: naomi.maruyama at noaa.gov
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Naomi Maruyama
>>>>> CIRES, Univ. of Colorado Boulder
>>>>> NOAA Space Weather Prediction Center
>>>>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>>>>> email: naomi.maruyama at noaa.gov
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Naomi Maruyama
>>> CIRES, Univ. of Colorado Boulder
>>> NOAA Space Weather Prediction Center
>>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>>> email: naomi.maruyama at noaa.gov
>>>
>>>
>>
>>
>> --
>>
>> Naomi Maruyama
>> CIRES, Univ. of Colorado Boulder
>> NOAA Space Weather Prediction Center
>> phone: +1 303.497.4857 <%28303%29%20497-4857>
>> email: naomi.maruyama at noaa.gov
>>
>>
>
>
> --
>
> Naomi Maruyama
> CIRES, Univ. of Colorado Boulder
> NOAA Space Weather Prediction Center
> phone: +1 303.497.4857 <%28303%29%20497-4857>
> email: naomi.maruyama at noaa.gov
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/wam-ipe/attachments/20170727/de225769/attachment-0001.html 