[Ncep.list.nems.announce] nems r86251: Updates to testing and documentation.
Samuel.Trahan at noaa.gov
Samuel.Trahan at noaa.gov
Wed Jan 4 17:43:05 UTC 2017
Test script
upd...
Message-ID: <586d3429.GqMbjsV9cjeINpjP%Samuel.Trahan at noaa.gov>
User-Agent: Heirloom mailx 12.4 7/29/08
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="=_586d3429.S3jI1KttufaePb/lYLrn9r+eNE0Z0uWmcVxwnWspmG+zYPeH"
This is a multi-part message in MIME format.
--=_586d3429.S3jI1KttufaePb/lYLrn9r+eNE0Z0uWmcVxwnWspmG+zYPeH
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Friendly NEMS developers,
This is an automated email about a NEMS commit.
Project: nems
URL: https://svnemc.ncep.noaa.gov/projects/nems/trunk
Revision: 86251
Author: samuel.trahan at noaa.gov
Date: 2017-01-04T17:38:45.681535Z
Message:
Updates to testing and documentation.
Test script updates:
1. Scripts will use the least full ptmp on WCOSS phase 1.
2. The test suite now verifies that the executable did not change
during the test executions.
3. Test reports include Subversion information.
4. The "rt.sh" wrapper around rtgen can send the requested project for
core-hours (-p option)
5. Fix for a bug in project detection (much thanks to Valery Yudin for
reporting this and testing the fix)
Documentation updates:
1. Mentions of NEMSLegacy are removed from all apps and NEMS framework
2. GSM-related pages do not mention NMM and vice-versa
3. All NMM references are removed from NEMSGSM and WAM-IPE apps
See attached file for full differences.
First 4000 bytes of differences:
Index: checkout/tests/rtgen
===================================================================
--- checkout/tests/rtgen (revision 85899)
+++ checkout/tests/rtgen (revision 86251)
@@ -9,6 +9,7 @@
import shutil
import getopt
import StringIO
+import socket
def fail(how):
"""!Aborts the program with status 1. Should only be called
@@ -172,7 +173,6 @@
import produtil.setup
except ImportError as ie:
altpath=os.path.join(os.path.dirname(os.path.realpath(__file__)),'produtil/ush')
- print altpath
if not os.path.isdir(altpath):
fail('%s is missing and produtil is not in PYTHONPATH. Is your produtil external missing?'%(altpath,))
sys.path.append(altpath)
@@ -295,7 +295,9 @@
batchexe('account_params'),logger=jlogger)
except(EnvironmentError,ExitStatusException) as ee:
logger.warning('Cannot run account_params: '+str(ee))
+ logger.warning('Will use project "nems" for cpu hours.')
return 'nems'
+ default_project='nems'
projects=list()
projalloc=dict()
for line in account_params.splitlines():
@@ -309,6 +311,8 @@
try:
alloc=float(alloc)
if name=='nems': alloc/=2
+ if not projects:
+ default_project=name
projects.append(name)
projalloc[name]=alloc
except (ValueError,TypeError) as vte:
@@ -316,14 +320,14 @@
continue
if not projects:
# Parse error or failure of account_params.
- logger.warning('Could not parse account_params output.')
- return 'nems'
+ logger.warning('Could not parse account_params output. Will use default: '+default_project)
+ return default_project
projects.sort(lambda a,b: cmp(projalloc[a],projalloc[b]))
projchoose=projects[-1]
if projalloc[projchoose]<1.0:
- logger.warning('All projects passed core-hour limit; will use nems.')
- return 'nems'
+ logger.warning('All projects passed core-hour limit; will use first project: '+default_project)
+ return default_project
for proj in projects:
if proj==projchoose:
@@ -413,8 +417,71 @@
def decide_tmp_wcoss1():
"""!Placeholder for future development; returns "/ptmpp/$USER"
where $USER is the username """
- return os.path.join('/ptmpp1',username())
+ logger=logging.getLogger('rtgen')
+ # are we on tide or gyre?
+ host=socket.gethostname()
+ tg=host[0] # = t for tide or g for gyre
+
+ ptmps=[ [ '/ptmpd1', '-j', 'ptmp-d1', 'gpfs-'+tg+'d1'],
+ [ '/ptmpd2', '-j', 'ptmp-d2', 'gpfs-'+tg+'d2'],
+ [ '/ptmpp1', '-j', 'ptmp-p1', 'gpfs-'+tg+'p1'] ]
+
+ # Area with maximum space available and available space in TB:
+ max_area='/ptmpp1' # default value on failure
+ max_avail=0
+
+ for ptmp in ptmps:
+ try:
+ args=['/usr/lpp/mmfs/bin/mmlsquota', '--block-size', '1T' ]
+ args.extend(ptmp[1:])
+ area=ptmp[0]
+ cmd=batchexe(args[0])[args[1:]]
+ mmlsquota=produtil.run.runstr(cmd,logger=logger)
+ if not mmlsquota:
+ logger.error('mmlsquota printed nothing')
+ continue
+
+ #gpfs-gd1 FILESET 19 147 147 1 none | 1399932 0 0 158 none
+ # ^ ^
+ # | +--- TB Limit
+ # +---------- TB Used
+
+ for m in re.finditer(r'''(?isx)
+ (?:
+ \S+ \s+ FILESET
+ \s+ (?P<TBused> \d+ )
+ \s+ (?P<TBquota> \d+ )
+ \s+ (?P<TBlimit> \d+ )
+ [^\r\n]* (?: [\r\n] | [\r\n]*\Z )
+ |
+ (?P<bad> [^\r\n]*[\r\n] | [^\r\n]*\Z )
+ )
+ ''',mmlsquota):
+
+ if m.group('bad') or not m.group('TBused') \
+ or not m.group('TBlimit'):
+
... see attachment for the rest ...
--=_586d3429.S3jI1KttufaePb/lYLrn9r+eNE0Z0uWmcVxwnWspmG+zYPeH
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="86251.diff"
Index: checkout/tests/rtgen
===================================================================
--- checkout/tests/rtgen (revision 85899)
+++ checkout/tests/rtgen (revision 86251)
@@ -9,6 +9,7 @@
import shutil
import getopt
import StringIO
+import socket
def fail(how):
"""!Aborts the program with status 1. Should only be called
@@ -172,7 +173,6 @@
import produtil.setup
except ImportError as ie:
altpath=os.path.join(os.path.dirname(os.path.realpath(__file__)),'produtil/ush')
- print altpath
if not os.path.isdir(altpath):
fail('%s is missing and produtil is not in PYTHONPATH. Is your produtil external missing?'%(altpath,))
sys.path.append(altpath)
@@ -295,7 +295,9 @@
batchexe('account_params'),logger=jlogger)
except(EnvironmentError,ExitStatusException) as ee:
logger.warning('Cannot run account_params: '+str(ee))
+ logger.warning('Will use project "nems" for cpu hours.')
return 'nems'
+ default_project='nems'
projects=list()
projalloc=dict()
for line in account_params.splitlines():
@@ -309,6 +311,8 @@
try:
alloc=float(alloc)
if name=='nems': alloc/=2
+ if not projects:
+ default_project=name
projects.append(name)
projalloc[name]=alloc
except (ValueError,TypeError) as vte:
@@ -316,14 +320,14 @@
continue
if not projects:
# Parse error or failure of account_params.
- logger.warning('Could not parse account_params output.')
- return 'nems'
+ logger.warning('Could not parse account_params output. Will use default: '+default_project)
+ return default_project
projects.sort(lambda a,b: cmp(projalloc[a],projalloc[b]))
projchoose=projects[-1]
if projalloc[projchoose]<1.0:
- logger.warning('All projects passed core-hour limit; will use nems.')
- return 'nems'
+ logger.warning('All projects passed core-hour limit; will use first project: '+default_project)
+ return default_project
for proj in projects:
if proj==projchoose:
@@ -413,8 +417,71 @@
def decide_tmp_wcoss1():
"""!Placeholder for future development; returns "/ptmpp/$USER"
where $USER is the username """
- return os.path.join('/ptmpp1',username())
+ logger=logging.getLogger('rtgen')
+ # are we on tide or gyre?
+ host=socket.gethostname()
+ tg=host[0] # = t for tide or g for gyre
+
+ ptmps=[ [ '/ptmpd1', '-j', 'ptmp-d1', 'gpfs-'+tg+'d1'],
+ [ '/ptmpd2', '-j', 'ptmp-d2', 'gpfs-'+tg+'d2'],
+ [ '/ptmpp1', '-j', 'ptmp-p1', 'gpfs-'+tg+'p1'] ]
+
+ # Area with maximum space available and available space in TB:
+ max_area='/ptmpp1' # default value on failure
+ max_avail=0
+
+ for ptmp in ptmps:
+ try:
+ args=['/usr/lpp/mmfs/bin/mmlsquota', '--block-size', '1T' ]
+ args.extend(ptmp[1:])
+ area=ptmp[0]
+ cmd=batchexe(args[0])[args[1:]]
+ mmlsquota=produtil.run.runstr(cmd,logger=logger)
+ if not mmlsquota:
+ logger.error('mmlsquota printed nothing')
+ continue
+
+ #gpfs-gd1 FILESET 19 147 147 1 none | 1399932 0 0 158 none
+ # ^ ^
+ # | +--- TB Limit
+ # +---------- TB Used
+
+ for m in re.finditer(r'''(?isx)
+ (?:
+ \S+ \s+ FILESET
+ \s+ (?P<TBused> \d+ )
+ \s+ (?P<TBquota> \d+ )
+ \s+ (?P<TBlimit> \d+ )
+ [^\r\n]* (?: [\r\n] | [\r\n]*\Z )
+ |
+ (?P<bad> [^\r\n]*[\r\n] | [^\r\n]*\Z )
+ )
+ ''',mmlsquota):
+
+ if m.group('bad') or not m.group('TBused') \
+ or not m.group('TBlimit'):
+ logger.debug('mmlsquota: ignoring %s'%(
+ repr(m.group(0).strip()),))
+ continue
+ avail=int(m.group('TBlimit')) - int(m.group('TBused'))
+ logger.info('%s: %d TB available'%(area,avail))
+ if avail>max_avail:
+ logger.info('Higher than %s: %d TB available'%(max_area,max_avail))
+ ( max_area, max_avail) = ( area, avail )
+ except(EnvironmentError,ExitStatusException,KeyError,ValueError) as e:
+ # Log all likely errors before emergency fallback option:
+ logger.error(str(e),exc_info=True)
+
+ if max_area:
+ logger.info('%s: use this ptmp with %d TB available'%(
+ max_area,max_avail))
+ else:
+ logger.warning('Auto-detection of least used ptmp failed.')
+ logger.warning('Will fall back to %s'%(max_area,))
+
+ return os.path.join(max_area,username())
+
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Custom test generator:
@@ -422,7 +489,7 @@
def __init__(self,baseline,scratch_dir,unique_id=None,
logger=None,baseline_dir=None,
verbose=True,dry_run=False,inputfile=None,
- setarith=None):
+ setarith=None,project=None):
baseline=bool(baseline)
self.no_copy_template = baseline_dir is not None
if unique_id is None:
@@ -440,11 +507,20 @@
self._new_baseline=os.path.join(
self._scratch_dir,'REGRESSION_TEST')
self.platform_name=None
+ self.project=project
+ assert(project)
+ assert(self.project)
def override(self,scope):
+ assert(self.project)
self._scope=scope
- if not self._new_baseline:
- return
- scope.override_local([scope],'plat%BASELINE',self._new_baseline)
+ if self._new_baseline:
+ scope.override_local([scope],'plat%BASELINE',self._new_baseline)
+ if self.project:
+ self.logger.warning('override project with '+self.project)
+ scope.override_local([scope],'plat%CPU_ACCOUNT',self.project)
+ scope.override_local([scope],'plat%ACCOUNT',self.project)
+ else:
+ raise Exception('no project')
@property
def new_baseline(self):
return self._new_baseline
@@ -500,10 +576,9 @@
out=StringIO.StringIO()
self.make_bash_load_rocoto(out)
out.write(r'''
-set -x
rocotostat -w workflow.xml -d workflow.db -c ALL > rocotostat.txt
timestamp=$( ls -l --time=c --time-style=+%%s workflow.xml | awk '{print $6}' )
-%s/rtreportimpl ../com rocotostat.txt txt $timestamp > rtreport.txt
+%s/rtreportimpl ../com rocotostat.txt "${1:-txt}" $timestamp > rtreport.txt
cat rtreport.txt
'''%(bashify_string(os.path.realpath(os.path.dirname(__file__))),))
self.make_rtscript(self.outloc,'rtreport',out.getvalue())
@@ -669,7 +744,7 @@
testgen=RTGen(baseline,scratch_dir,unique_id,logger,
baseline_dir,inputfile=inputfile,
verbose=bool(verbose),dry_run=dry_run,
- setarith=arith)
+ setarith=arith,project=project)
try:
jlogger.info('Generating workflow with id %s.'%(repr(unique_id),))
testgen.testgen()
Index: checkout/tests/rtreportimpl
===================================================================
--- checkout/tests/rtreportimpl (revision 85899)
+++ checkout/tests/rtreportimpl (revision 86251)
@@ -2,22 +2,91 @@
import re, os, sys, time
+# Find the produtil package and load produtil.setup:
+try:
+ import produtil.setup
+except ImportError as ie:
+ altpath=os.path.join(os.path.dirname(os.path.realpath(__file__)),'produtil/ush')
+ if not os.path.isdir(altpath):
+ fail('%s is missing and produtil is not in PYTHONPATH. Is your produtil external missing?'%(altpath,))
+ sys.path.append(altpath)
+ import produtil.setup
+
+produtil.setup.setup(send_dbn=False,ologlevel=None)
+
+import produtil.cd
+from produtil.run import exe,runstr,run,alias,ExitStatusException
+
+########################################################################
+
+# Parse arguments
comdir=sys.argv[1]
stattxt=sys.argv[2]
mode=sys.argv[3]
timestamp=None
+just_status=False
if len(sys.argv)>4:
timestamp=int(sys.argv[4])
-assert(mode=='txt')
+if mode=='status':
+ just_status=True
+elif mode!='txt':
+ sys.stderr.write('Warning: output mode %s is not supported. I will '
+ 'assume you meant "txt"'%(mode,))
+
+def rtreport(line=''):
+ if not just_status:
+ print line
+def rtstatus(line,also_report=True):
+ if also_report or just_status:
+ print line
+
+def get_md5(md5file):
+ with open(md5file,'rt') as f:
+ line=f.readline()
+ split=line.strip().split()
+ if not split: return (None,None)
+ if len(split)<2:
+ return (split[0],None)
+ else:
+ return (split[0],split[1])
+
+if timestamp is not None:
+ rtreport('WORKFLOW STARTED AT %s (+%d)'%(
+ time.ctime(timestamp),timestamp))
+
+checkout_top=os.path.dirname(os.path.dirname(os.path.dirname(
+ os.path.realpath(__file__))))
+
+rtreport( 'Repository information:')
+rtreport()
+rtreport()
+rtreport( 'REPO TOP:')
+svn=alias(exe('svn'))
+info=alias(svn['info'])
+try:
+ with produtil.cd.NamedDir(checkout_top):
+ rtreport(runstr(info['.']))
+ status=runstr(svn['status',checkout_top])
+ for line in status.splitlines():
+ m=re.match('X\s+(.*)',line)
+ if not m:
+ continue
+ rtreport('EXTERNAL %s:'%(m.group(1),))
+ rtreport(runstr(info[os.path.join(checkout_top,m.group(1))]))
+except(ValueError,KeyError,ExitStatusException,EnvironmentError) as e:
+ print 'WARNING: Could not complete subversion checks: '+str(e)
+
itest=0
success=True
finished=True
+build_count=0
+tests_passed=0
+tests_failed=0
+builds_failed=0
+builds_passed=0
with open(stattxt,'rt') as f:
- if timestamp is not None:
- print 'WORKFLOW STARTED AT %s (+%d)'%(
- time.ctime(timestamp),timestamp)
for line in f:
m=re.match(r'''(?isx)
(?P<cycle> \d+ )
@@ -37,59 +106,128 @@
success=False
if name.find('build_')==0:
+ build_count+=1
if state == '-':
state='UNSTARTED'
- print 'BUILD %s: %s'%(name[6:],state)
+ rtreport('BUILD %s: %s'%(name[6:],state))
if state not in [ 'SUCCEEDED', 'DEAD', 'LOST' ]:
finished=False
+ if state == 'SUCCEEDED':
+ builds_passed+=1
+ elif state in ['DEAD','LOST']:
+ builds_failed+=1
+ rtstatus('BUILD %s: %s'%(name[6:],state),False)
continue
if name.find('test_')!=0:
- print 'UNKNOWN JOB %s: %s'%(name,state)
-
+ rtreport('UNKNOWN JOB %s: %s'%(name,state))
+
itest+=1
testname=name[5:] # part of name after "test_"
-
if state not in [ 'SUCCEEDED', 'DEAD', 'LOST' ]:
- print 'TEST #%d %s: not yet complete...'%(itest,testname)
+ rtreport('TEST #%d %s: not yet complete...'%(itest,testname))
finished=False
continue
reportfile=os.path.join(comdir,testname,'report.txt')
if not os.path.isfile(reportfile):
- print 'TEST #%d %s: FAIL\n No such file: %s'%(
- itest,testname,reportfile)
+ rtstatus('TEST #%d %s: FAIL\n No such file: %s'%(
+ itest,testname,reportfile))
+ tests_failed+=1
continue
test_passed=False
+ md5_passed=True
report=list()
+ pass_fail_lines=list()
+ md5local=None
+ md5ref=None
with open(reportfile,'rt') as f:
for reportline in f:
- if reportline.find('TEST PASSED')==0:
+ m=re.match(r'''(?x)
+ \s* md5sum: .*
+ | \s* md5sum \s+ local=(?P<md5local>.*)
+ | \s* md5sum \s+ reference=(?P<md5ref>.*)
+ ''',reportline)
+ if m:
+ if m.group('md5local'):
+ md5local=m.group('md5local')
+ elif m.group('md5ref'):
+ md5ref=m.group('md5ref')
+ elif reportline.find('TEST PASSED')!=-1:
test_passed=True
- report.append(' '+reportline.strip())
+ pass_fail_lines.append(' '+reportline.strip())
+ elif reportline.find('TEST FAIL')!=-1:
+ pass_fail_lines.append(' '+reportline.strip())
+ else:
+ if reportline.find('missing')>=0 or reportline.find('mismatch')>=0:
+ rtstatus(name+': '+reportline.strip(),False)
+ report.append(' '+reportline.strip())
+
+ if md5local and md5ref:
+ (localsum,localfile) = get_md5(md5local)
+ (refsum,reffile) = get_md5(md5ref)
+ md5local,md5ref=None,None
+ if localsum != refsum:
+ rtstatus('%s: %s changed since build'%(name,reffile),False)
+ report.append(''' FAIL: EXECUTABLE USED DOES NOT MATCH LAST BUILD.
+ File: %s
+ Expected md5sum: %s
+ Actual md5sum: %s'''%(reffile,refsum,localsum))
+ md5_passed=False
+ else:
+ report.append(''' Executable did not change during test suite:
+ File: %s
+ Expected md5sum: %s
+ Actual md5sum: %s'''%(reffile,refsum,localsum))
+
+ # The pass vs. fail judgement must include the md5sum checks:
+ test_passed=test_passed and md5_passed
+
+ # Lastly, it must also include the state information:
if state != 'SUCCEEDED':
+ rtreport(' FAIL: job state is '+state)
test_passed=False
- print 'TEST #%d: %s\n%s'%(
- itest,
- 'PASS' if test_passed else 'FAIL',
- '\n'.join(report))
+ success=success and test_passed
+
+ # If the job says the test passed, but the md5sum or job
+ # failed, we report the test as a failure:
+ for line in pass_fail_lines:
+ if not test_passed:
+ line=line.replace('TEST PASSED','TEST FAILED',1)
+ report.append(line)
+
+ rtreport()
+ if test_passed:
+ rtreport('TEST #%d: PASS'%itest)
+ tests_passed+=1
+ else:
+ rtreport('TEST #%d: FAIL'%itest)
+ rtstatus('%s: FAIL'%(name,),False)
+ tests_failed+=1
+ rtreport('\n'.join(report))
+
now=int(time.time())
- print 'WORKFLOW REPORT AT %s (+%d)'%(
- time.ctime(now),now)
+ rtreport('WORKFLOW REPORT AT %s (+%d)'%(
+ time.ctime(now),now))
+rtstatus('Tests: %d failed, %d passed out of %d'%(
+ tests_failed,tests_passed,itest))
+rtstatus('Builds: %d failed, %d passed out of %d'%(
+ builds_failed,builds_passed,build_count))
+
if not finished:
if not success:
- print 'REGRESSION TEST IS STILL RUNNING (SOME TESTS FAILED)'
+ rtstatus('REGRESSION TEST IS STILL RUNNING (SOME TESTS FAILED)')
exit(1)
else:
- print 'REGRESSION TEST IS STILL RUNNING (ALL TESTS PASSED SO FAR)'
+ rtstatus('REGRESSION TEST IS STILL RUNNING (ALL TESTS PASSED SO FAR)')
exit(0)
if success:
- print 'REGRESSION TEST WAS SUCCESSFUL'
+ rtstatus('REGRESSION TEST WAS SUCCESSFUL')
exit(0)
else:
- print 'REGRESSION TEST FAILED'
+ rtstatus('REGRESSION TEST FAILED')
exit(1)
Index: checkout/tests/rt.sh
===================================================================
--- checkout/tests/rt.sh (revision 85899)
+++ checkout/tests/rt.sh (revision 86251)
@@ -21,6 +21,7 @@
echo " -h display this help"
echo " -r PLATFORM:/path/to/run"
echo " rerun past suite without regenerating it"
+ echo " -p project = set the project to use for cpu time"
echo
echo "Common <subset>s: gfs, nmm, slg, wam, debug"
echo "See ../../compsets/all.input for a full list"
@@ -42,7 +43,7 @@
RUNDIR=''
PLATFORM_NAME=''
-while getopts ":c:fst:n:hr:" opt; do
+while getopts ":c:fst:n:hr:p:" opt; do
case $opt in
r)
if [[ $OPTARG =~ ^([a-zA-Z][a-zA-Z_.0-9]*):(.+)$ ]] ; then
@@ -54,6 +55,9 @@
usage "Rerun argument must be PLATFORM_NAME:RUNDIR"
fi
;;
+ p)
+ cmd="$cmd -p $OPTARG"
+ ;;
c)
if [[ ! -z "$set_info" ]] ; then
usage "Only one of -c, -s, -t, or -f can be used."
@@ -174,7 +178,7 @@
fi
cmd="$RUNDIR/rtreport > $REPORT"
echo "rt.sh: run $cmd"
- $cmd > "$REPORT"
+ "$RUNDIR/rtreport" > "$REPORT"
repstatus="$?"
echo "rt.sh: status $repstatus"
if [[ "$repstatus" == 0 ]] ; then
Index: checkout/tests
===================================================================
--- checkout/tests (revision 85899)
+++ checkout/tests (revision 86251)
Property changes on: checkout/tests
___________________________________________________________________
Modified: svn:externals
## -1 +1 ##
-produtil -r85398 https://svnemc.ncep.noaa.gov/projects/nceplibs/produtil/branches/regtests
+produtil -r86179 https://svnemc.ncep.noaa.gov/projects/nceplibs/produtil/branches/regtests
Modified: svn:mergeinfo
## -0,0 +0,1 ##
Merged /nems/branches/project/tests:r85958-86237
Index: checkout/doc/NEMS.md
===================================================================
--- checkout/doc/NEMS.md (revision 85899)
+++ checkout/doc/NEMS.md (revision 86251)
@@ -3,37 +3,42 @@
The NEMS directory contains the source code and test scripts for the
NEMS. Most of the documentation is in the `doc` subdirectory or in
-the `../doc/` directory.
+the `../doc/` directory. Most of the files that were in the NEMS have
+been moved to the application layer, discussed below. Further
+documentation, specific to the app, is also at the app level.
-What is NOT in the NEMS is the source code to the components, the
-choice of build environment and some aspects of running the system.
-Those parts of the NEMS are in a directory above this, at the
-so-called "app" level. Further documentation, specific to the app, is
-also at the app level.
-
Within NEMS resides:
* `exe` - NEMS.x and other executables built from `src`
* `src` - main program for NEMS
- * `atmos` - The ATM component code and its internal state. This will soon be removed
- * `gfs` - All files directly related to the global spectal model
- * `nmm` - All files related to the non-hydrostatic multi-scale model
- * `fim` - all files related to the FIM
- * `gen` - All files related to the GEN
- * `share` - Assorted source shared by all model cores.
* `ENS_Cpl` - The Ensemble coupler directory.
- * `post` - contains subroutines to conenct NEMS and the Unified Post
- Processor (UPP), no UPP source code included
- * `chem/gocart` - The GOCART source files.
* `conf` - various compliation specifications
-
-* `tests` - test execution scripts. Most of these will soon be removed or
- moved to the app level.
-* `oldtests` - the old test suite which is deprecated but retained for
- backward compatibility
-* `compsets` - configuration for the NEMSCompsetRun. This will soon be
- moved to the app level.
* `doc` - documentation.
* `NEMSAppBuilder` - a script to build NEMS, as discussed elsewhere in the
documentation
-* `NEMSCompsetRun` - script to run NEMS
\ No newline at end of file
+* `NEMSCompsetRun` - script to run NEMS, identical to the regression test runner
+* `OldCompsetRun` - prior version of the compset runner
+* `tests` - test execution logic
+ * `rtgen` - front-end to the regression test runner
+ * `rt.sh` - wrapper around rtgen for users familiar with the old system
+
+At the application level resides these files:
+
+* `doc` - application-specific documentation
+
+* `oldtests` - application-specific, old, test suite which is
+ deprecated but retained for backward compatibility
+
+* `compsets` - configuration for the NEMSCompsetRun and regression
+ test runner
+
+* `oldcompsets` - configuration for the old compset system available
+ via OldCompsetRunner
+
+* `modulefiles` - module loading information for each platform
+ * `theia` - NOAA Theia modulefiles
+ * `wcoss.phase1` - WCOSS Phase 1 modulefiles
+ * ... other directories for other computers ...
+* `conf` - configuration for NEMS/src/configure
+* `parm` - parameter files for the test suites
+* `log` - log directory for the NEMSAppBuilder and NEMSCompsetRun
\ No newline at end of file
Index: checkout/doc/markdown.md
===================================================================
--- checkout/doc/markdown.md (revision 85899)
+++ checkout/doc/markdown.md (revision 86251)
@@ -12,7 +12,7 @@
This documentation is stored in the NEMSLegacy doc directory. The
website can be regenerated by doing this:
- svn co https://svnemc.ncep.noaa.gov/projects/nems/apps/NEMSLegacy/trunk
+ svn co https://svnemc.ncep.noaa.gov/projects/nems/apps/(appname)/trunk
cd trunk/NEMS/doc
make
Index: checkout/doc/NEWTEST.md
===================================================================
--- checkout/doc/NEWTEST.md (revision 85899)
+++ checkout/doc/NEWTEST.md (revision 86251)
@@ -70,6 +70,9 @@
when running the suite in verification or baseline generation modes.
* `-r PLATFORM:/path/to/rtgen.###` - used by the full test method.
See below.
+* `-p project` = set the project or account to use for CPU hours.
+ If unspecified, one will be automatically picked based on
+ cpu availability.
### Full Test Method
@@ -162,12 +165,21 @@
At that point, you can run rtreport to get a report of the tests.
Actually, you can run rtreport at any time. If the tests are not yet
complete, it will tell you which ones are complete. It will report
-all it knows about failed tests too.
+all it knows about failed tests too. There are two output formats:
- /path/to/USERNAME/rtgen.23768/rtreport
+To run:
-The output looks something like this
+ /path/to/USERNAME/rtgen.23768/rtreport [mode]
+Where the optional `mode` is one of:
+
+ * `status` - short output that only lists failed tests and counts
+ the number of failed, complete, and unfinished tests.
+
+ * `txt` - full text output of all information (the default).
+
+The output of `txt` mode (the default) looks something like this
+
BUILD nmm.x: SUCCEEDED
BUILD nmm.debug.x: SUCCEEDED
BUILD gsm.x: SUCCEEDED
@@ -189,7 +201,23 @@
... information about more tests ...
+### <a name="rerun"></a>Rerunning Failed Tests
+If a test fails, you can request that it be rerun via the `rtrewind`
+command. The command is located in the same directory as `rtrun`
+and can be called in two different ways:
+
+ /path/to/USERNAME/rtgen.23768/rtrewind -a
+
+ /path/to/USERNAME/rtgen.23768/rtrewind job1 [job2 [...]]
+
+The first method requests a rerun of ALL tests and builds while the
+second requests only certain ones be rerun.
+
+The jobs (`job1`, `job2`, ...) are the names from the test suite such
+as `gsm.x` or `nmm_cntrl`. You can optionally include `test_` or
+`build_` before the name, as it is printed by the `rtreport` command.
+
### <a name="run-sub"></a>Running Subsets of the Test Suite
The test suite, as of this writing, has 48 tests and 5 build options.
Index: checkout
===================================================================
--- checkout (revision 85899)
+++ checkout (revision 86251)
Property changes on: checkout
___________________________________________________________________
Modified: svn:mergeinfo
## -0,0 +0,1 ##
Merged /nems/branches/project:r85958-86237
--=_586d3429.S3jI1KttufaePb/lYLrn9r+eNE0Z0uWmcVxwnWspmG+zYPeH--
More information about the Ncep.list.nems.announce
mailing list