[Ncep.list.nems.announce] Planned commit to NEMS trunk: (step 3 under NEMS ticket #41)

[Yusong Wang - NOAA Affiliate]

Ratko, Moorthi and others,

We did implement wrappers for both qstat and qsub to work around the PBS
issues and replaced the default version during the maintenance.

Under normal conditions, the new wrappers are much more stable than the
original qsub and qstat. While as Ratko pointed out, there are massive
number of jobs submitted in the last couple of days, which makes the system
(especially the Moab/Torque) over-loaded. We are working with the vendor to
find a feasible way to prevent this from happening in the future.

The Zeus team has been working with the user to reduce the workload in the
last couple of days. We have observed the throughput of PBS in the last 24
hours has been improved significantly due to the reduced workload on the
system.

I am running a previous version of NEMS regression test on Zeus this
morning. So far, 11 tests passed without any issue. Please give another try
to see if it is any better today.


Thanks for your patience.



On Wed, Sep 18, 2013 at 11:19 AM, Ratko Vasic <ratko.vasic at noaa.gov> wrote:

> This is PBS error. I thought they solved problem: there was user
> submitting several thousands of jobs at same time(more than 6k).Now I
> see only ~1500 jobs on Hold (from same user).
>
> Ratko
>
> On 9/18/2013 7:29 AM, Shrinivas Moorthi wrote:
> > pbs_iff: Invalid credential MSG=cannot authenticate user. Client
> > connection not found
> > No Permission.
>
> --
> Ratko Vasic
> Meteorologist
> 301-683-3814
> National Oceanic and Atmospheric Administration
> NCEP/EMC,  Room 2791
> NCWCP  W/NP2
> 5830 University Research Court
> College Park, MD  20740-3818
>
> _______________________________________________
> Ncep.list.nems.announce mailing list
> Ncep.list.nems.announce at lstsrv.ncep.noaa.gov
> https://lstsrv.ncep.noaa.gov/mailman/listinfo/ncep.list.nems.announce
>



-- 
Yusong Wang,   Ph.D.
High Performance Computing Application Specialist
NOAA/ National Weather Service
National Centers for Environmental Prediction

Building: NCWCP, Room: 2028
5830 University Research Ct
College Park,MD   20740
Tel (Office):  (301)683-3690
Fax: (301)683-3703
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lstsrv.ncep.noaa.gov/pipermail/ncep.list.nems.announce/attachments/20130918/c268a3ed/attachment.html