[WAM-IPE] Realtime wam-ipe runs on theia: archive
Rashid Akmaev - NOAA Federal
rashid.akmaev at noaa.gov
Tue Jul 25 23:32:16 UTC 2017
Is it possible to catch when a tube first fails? Does the restart from MSIS Houjun uses make them fail? Plotting composition after the fact probably may not help much as failed tubes will most likely stay that way no matter what the composition, right?
Rashid Akmaev
> On Jul 25, 2017, at 4:25 PM, Naomi Maruyama - NOAA Affiliate <naomi.maruyama at noaa.gov> wrote:
>
> Hi Houjun,
>
> Thanks for your showing some updates at our meeting. Your results look very encouraging. But could you please also plot the WAM fields from your runs in order to confirm that the WAM fields that are used in IPE are reasonable?
>
> In your plotting script, set the VarType to 4 (instead of 7 for NmF2) if you want to plot WAM neutral atomic Oxygen, for example. Also you could change the altitude of your plot by changing the line, "setenv alt" inside the visualization script.
>
> Thanks.
> naomi
>
>
>
>
>
>
>> On Tue, Jul 25, 2017 at 2:29 PM, Naomi Maruyama - NOAA Affiliate <naomi.maruyama at noaa.gov> wrote:
>> Thanks.
>>
>> I see that failed flux tubes in your plots and FLIP_ERR in your runs. But I cannot suggest anything to solve the problem until I can make some test runs.
>>
>> In order to solve this issue, I would need to reproduce your run by myself, and try to debug the run. Could you please suggest how I can do that?
>>
>> Thanks.
>> naomi
>>
>>> On Tue, Jul 25, 2017 at 10:30 AM, Houjun Wang - NOAA Affiliate <houjun.wang at noaa.gov> wrote:
>>> yes
>>>
>>> Houjun Wang
>>>
>>>> On Tue, Jul 25, 2017 at 10:25 AM, Naomi Maruyama - NOAA Affiliate <naomi.maruyama at noaa.gov> wrote:
>>>> Houjun
>>>>
>>>> Thank you very much for archiving the files. I am going to study them.
>>>>
>>>>
>>>>
>>>> >>I have saved the entire rundir for two runs in:
>>>>
>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071606gdasfcst1
>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071612gdasfcst1
>>>>
>>>> you mentioned at the previous meeting that IPE is slowing down with failed flux tubes. Are these two runs have the problem?
>>>>
>>>> Thanks.
>>>> naomi
>>>>
>>>>> On Mon, Jul 24, 2017 at 4:45 PM, Houjun Wang - NOAA Affiliate <houjun.wang at noaa.gov> wrote:
>>>>> In /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe
>>>>>
>>>>> 1. The following are already saved:
>>>>> plasmaXX:
>>>>> ut_rec
>>>>> IPE.inp
>>>>> stupXX
>>>>> Note: Since stup* files are from previous run, they are already saved in the previous (6 hours earlier) run dir. The soft link in the current run dir are for the NEXT run. I will remove these soft link (and change my run scripts accordingly), because this can be confusing.
>>>>>
>>>>> 2. The following will be saved in the future:
>>>>> FLIP_ERR
>>>>> logfile_input_params.log
>>>>>
>>>>> 3. The following are saved to mass storage (HPSS): they are called 'dayfile' which contain both output:
>>>>> standard output
>>>>> standard error output
>>>>>
>>>>> I have saved the entire rundir for two runs in:
>>>>>
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071606gdasfcst1
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071612gdasfcst1
>>>>>
>>>>> and the dayfile:
>>>>>
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071606gdasfcst1_0.dayfile
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/wam2017071612gdasfcst1_0.dayfile
>>>>>
>>>>> and the figs of NmF2 in
>>>>>
>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/fig/
>>>>>
>>>>> Let me know if I missed anything.
>>>>>
>>>>> (I will compare NmF2 in the forecast and in the initial conditions used for restart.)
>>>>>
>>>>>
>>>>> Houjun Wang
>>>>>
>>>>>> On Thu, Jul 20, 2017 at 1:23 PM, Naomi Maruyama - NOAA Affiliate <naomi.maruyama at noaa.gov> wrote:
>>>>>> Houjun and Adam,
>>>>>>
>>>>>> here is a list of files I would like to suggest to be saved under your archive(/scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe)
>>>>>>
>>>>>> plasmaXX
>>>>>> ut_rec
>>>>>> IPE.inp
>>>>>> stupXX
>>>>>> FLIP_ERR
>>>>>> logfile_input_params.log
>>>>>> standard output
>>>>>> standard error output
>>>>>>
>>>>>>
>>>>>> Theses files would be helpful if you want me to look at your runs and verify the results.
>>>>>>
>>>>>> Thanks.
>>>>>> naomi
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Wed, Jul 12, 2017 at 2:30 PM, Adam Kubaryk - NOAA Affiliate <adam.kubaryk at noaa.gov> wrote:
>>>>>>> Hi Naomi and Houjun,
>>>>>>>
>>>>>>> I'll chime in quickly on the parallel script workflow.
>>>>>>>
>>>>>>> That para_config file is used to submit a job that:
>>>>>>>
>>>>>>> runs /scratch3/NCEPDEV/swpc/svn/gfs/branches/gfs_workflow.v2.0.0/para/jobs/fcst.sh
>>>>>>> fcst.sh immediately calls the para_config before running anything else
>>>>>>> para_config then calls /scratch3/NCEPDEV/swpc/svn/gfs/branches/gfs_workflow.v2.0.0/para/ush/reconcile.sh
>>>>>>> after reconcile.sh, para_config finishes as well, and then fcst.sh runs
>>>>>>> fcst.sh calls /scratch3/NCEPDEV/swpc/svn/gfs/branches/global_shared.v14.0.0/scripts/exglobal_fcst_nems.sh.ecf-wam-ipe_wdas_iau2
>>>>>>>
>>>>>>> A lot of what's relevant to IPE itself (copying in or linking files to the RUNDIR, setting up the namelists, etc.) is going to occur in that final exglobal script.
>>>>>>>
>>>>>>> Let me know if there's anything else I can clarify.
>>>>>>>
>>>>>>> Best,
>>>>>>> Adam
>>>>>>>
>>>>>>>> On Wed, Jul 12, 2017 at 4:22 PM, Houjun Wang - NOAA Affiliate <houjun.wang at noaa.gov> wrote:
>>>>>>>> 1. Yes, we have hourly output from ipe and since this WDAS-IAU cycled runs, each run only have 12 hour duration (and since ipe doesn't output the last time step - George know this well and may have a fix for it now - it does't have 15 UT for example).
>>>>>>>>
>>>>>>>> So this is not an issue.
>>>>>>>> 2. The source code is in
>>>>>>>>
>>>>>>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU
>>>>>>>>
>>>>>>>> see especially the mods in
>>>>>>>>
>>>>>>>> /scratch3/NCEPDEV/swpc/svn/nems/WAM-IPE_NEMS201606_IAU/IPELIB/src/plasma/io_plasma_bin.f90
>>>>>>>>
>>>>>>>> for initializing ipe from ipe output from previous cycle in WDAS-IAU.
>>>>>>>>
>>>>>>>> I added another option "sw_record_number=2" for this, see IPE.inp for more details and other changes. I also reset record number to zero for each run in WDAS-IAU cycles to make it more practical for this purpose.
>>>>>>>>
>>>>>>>> 3. Now, since I am using EMC para workflow, there is not just one script.
>>>>>>>>
>>>>>>>> The config file I am using is:
>>>>>>>>
>>>>>>>>
>>>>>>>> /scratch3/NCEPDEV/swpc/save/swpc.spacepara/para/para_config_wam_ipe_wdas_iau
>>>>>>>>
>>>>>>>> which defines all the runs.
>>>>>>>>
>>>>>>>> Kate wrote some notes on using EMC para scripts/workflow for wam/wdas:
>>>>>>>>
>>>>>>>> https://docs.google.com/a/noaa.gov/document/d/1OmlGZc0WQy8kdjdPWqHLQZUVn47fXqTn41rbgyrCguc/edit?usp=sharing
>>>>>>>>
>>>>>>>> l'll see if I can answer if you have questions.
>>>>>>>>
>>>>>>>>
>>>>>>>> Houjun Wang
>>>>>>>>
>>>>>>>>> On Wed, Jul 12, 2017 at 1:24 PM, Naomi Maruyama - NOAA Affiliate <naomi.maruyama at noaa.gov> wrote:
>>>>>>>>> Houjun,
>>>>>>>>>
>>>>>>>>> Thank you very much for your kindly sharing your output files.
>>>>>>>>>
>>>>>>>>> I have a feeling that your IPE output files were generated from only 3UT to 14UT only for each day, after looking at some of your IPE ARCHIVE directory.
>>>>>>>>>
>>>>>>>>> In order to understand this potential issue in further detail, I would need to understand how your restart of IPE has been set up in your script and source code. Would you mind sharing your script and source code with which you have generated these output files?
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>> naomi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Tue, Jul 11, 2017 at 3:49 PM, Houjun Wang - NOAA Affiliate <houjun.wang at noaa.gov> wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> Some info on realtime wam-ipe runs on theia:
>>>>>>>>>>
>>>>>>>>>> 1) RUNDIR (this is temporary dir and it could be gone anytime)
>>>>>>>>>>
>>>>>>>>>> /scratch4/NCEPDEV/stmp4/swpc.spacepara/wam2017070506gdasfcst1
>>>>>>>>>>
>>>>>>>>>> 2) ARCHIVE (not all files are saved for ipe; what else should be saved?)
>>>>>>>>>>
>>>>>>>>>> /scratch3/NCEPDEV/swpc/noscrub/swpc.spacepara/archive/ipe
>>>>>>>>>>
>>>>>>>>>> 3) HPSS ARCHIVE (George can get ICs for WAM here):
>>>>>>>>>>
>>>>>>>>>> /5year/NCEPDEV/swpc-para/swpc.spacepara/THEIA/prwam_ipe
>>>>>>>>>>
>>>>>>>>>> Let me know if you have any questions or suggestions.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>>
>>>>>>>>>> Houjun Wang
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Naomi Maruyama
>>>>>>>>> CIRES, Univ. of Colorado Boulder
>>>>>>>>> NOAA Space Weather Prediction Center
>>>>>>>>> phone: +1 303.497.4857
>>>>>>>>> email: naomi.maruyama at noaa.gov
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> WAM-IPE mailing list
>>>>>>>> WAM-IPE at lstsrv.ncep.noaa.gov
>>>>>>>> https://www.lstsrv.ncep.noaa.gov/mailman/listinfo/wam-ipe
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Naomi Maruyama
>>>>>> CIRES, Univ. of Colorado Boulder
>>>>>> NOAA Space Weather Prediction Center
>>>>>> phone: +1 303.497.4857
>>>>>> email: naomi.maruyama at noaa.gov
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Naomi Maruyama
>>>> CIRES, Univ. of Colorado Boulder
>>>> NOAA Space Weather Prediction Center
>>>> phone: +1 303.497.4857
>>>> email: naomi.maruyama at noaa.gov
>>>
>>
>>
>>
>> --
>> Naomi Maruyama
>> CIRES, Univ. of Colorado Boulder
>> NOAA Space Weather Prediction Center
>> phone: +1 303.497.4857
>> email: naomi.maruyama at noaa.gov
>
>
>
> --
> Naomi Maruyama
> CIRES, Univ. of Colorado Boulder
> NOAA Space Weather Prediction Center
> phone: +1 303.497.4857
> email: naomi.maruyama at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://www.lstsrv.ncep.noaa.gov/pipermail/wam-ipe/attachments/20170725/2053e7d8/attachment-0001.html
More information about the WAM-IPE
mailing list