[Ncep.list.nems.announce] Planned commit to NEMS trunk: (step 3 under NEMS ticket #41)

Raghu Reddy raghu.reddy at noaa.gov
Wed Sep 18 15:48:15 UTC 2013

It so happens I was also doing the same thing!  And it has completed seven
tests so far and seems to be progressing well.


And as Yusong has explained, we are working on different ways of addressing
this issue.








From: ncep.list.nems.announce-bounces at lstsrv.ncep.noaa.gov
[mailto:ncep.list.nems.announce-bounces at lstsrv.ncep.noaa.gov] On Behalf Of
Yusong Wang - NOAA Affiliate
Sent: Wednesday, September 18, 2013 11:38 AM
To: Ratko Vasic
Cc: ncep.list.nems.announce at lstsrv.ncep.noaa.gov
Subject: Re: [Ncep.list.nems.announce] Planned commit to NEMS trunk: (step 3
under NEMS ticket #41)


Ratko, Moorthi and others,

We did implement wrappers for both qstat and qsub to work around the PBS
issues and replaced the default version during the maintenance. 

Under normal conditions, the new wrappers are much more stable than the
original qsub and qstat. While as Ratko pointed out, there are massive
number of jobs submitted in the last couple of days, which makes the system
(especially the Moab/Torque) over-loaded. We are working with the vendor to
find a feasible way to prevent this from happening in the future.

The Zeus team has been working with the user to reduce the workload in the
last couple of days. We have observed the throughput of PBS in the last 24
hours has been improved significantly due to the reduced workload on the

I am running a previous version of NEMS regression test on Zeus this
morning. So far, 11 tests passed without any issue. Please give another try
to see if it is any better today.


Thanks for your patience.



On Wed, Sep 18, 2013 at 11:19 AM, Ratko Vasic <ratko.vasic at noaa.gov> wrote:

This is PBS error. I thought they solved problem: there was user
submitting several thousands of jobs at same time(more than 6k).Now I
see only ~1500 jobs on Hold (from same user).


On 9/18/2013 7:29 AM, Shrinivas Moorthi wrote:
> pbs_iff: Invalid credential MSG=cannot authenticate user. Client
> connection not found
> No Permission.

Ratko Vasic
National Oceanic and Atmospheric Administration
NCEP/EMC,  Room 2791
5830 University Research Court
College Park, MD  20740-3818

Ncep.list.nems.announce mailing list
Ncep.list.nems.announce at lstsrv.ncep.noaa.gov

Yusong Wang,   Ph.D.
High Performance Computing Application Specialist
NOAA/ National Weather Service
National Centers for Environmental Prediction

Building: NCWCP, Room: 2028
5830 University Research Ct 
College Park,MD   20740 
Tel (Office):  (301)683-3690
Fax: (301)683-3703

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lstsrv.ncep.noaa.gov/pipermail/ncep.list.nems.announce/attachments/20130918/31e00a77/attachment.html 

More information about the Ncep.list.nems.announce mailing list