builder: fx-team-linux64-pgo slave: bld-linux64-spot-132 starttime: 1470702603.1 results: exception (4) buildid: 20160808173001 builduid: 4d7a8406f94d4fd1b56b1fb59e7f712a revision: 5ff96be046af749644327f75b21ce9437d766886 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.096198) ========= master: http://buildbot-master94.bb.releng.use1.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.096636) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.096905) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.114026) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.114415) ========= rm -rf scripts properties in dir /builds/slave/fx-team-l64-pgo-00000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-132.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-l64-pgo-00000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003415 ========= master_lag: 0.05 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:03.166045) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 2 secs) (at 2016-08-08 17:30:03.166368) ========= bash -c 'python /tools/checkouts/build-tools/buildfarm/utils/archiver_client.py mozharness --repo integration/fx-team --rev 5ff96be046af749644327f75b21ce9437d766886 --destination scripts --debug' in dir /builds/slave/fx-team-l64-pgo-00000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'python /tools/checkouts/build-tools/buildfarm/utils/archiver_client.py mozharness --repo integration/fx-team --rev 5ff96be046af749644327f75b21ce9437d766886 --destination scripts --debug'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-132.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-l64-pgo-00000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2016-08-08 17:30:03,203 truncating revision to first 12 chars 2016-08-08 17:30:03,203 Setting DEBUG logging. 2016-08-08 17:30:03,203 attempt 1/10 2016-08-08 17:30:03,203 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/integration/fx-team/5ff96be046af?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2016-08-08 17:30:04,488 unpacking tar archive at: fx-team-5ff96be046af/testing/mozharness/ program finished with exit code 0 elapsedTime=1.953914 ========= master_lag: 0.10 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 2 secs) (at 2016-08-08 17:30:05.217775) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.218050) ========= script_repo_revision: 5ff96be046af749644327f75b21ce9437d766886 ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.218375) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.218650) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.229778) ========= ========= Started remove old nonce (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.230066) ========= rm -f nonce in dir /builds/slave/fx-team-l64-pgo-00000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'nonce'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-132.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-l64-pgo-00000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002973 ========= master_lag: 0.05 ========= ========= Finished remove old nonce (results: 0, elapsed: 0 secs) (at 2016-08-08 17:30:05.285005) ========= ========= Started download_token exception (results: 4, elapsed: 45 mins, 14 secs) (at 2016-08-08 17:30:05.285300) ========= Slave: bld-linux64-spot-132 IP: 10.134.55.251 Duration: 25200 URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing1.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing2.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing7.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing5.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing4.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing3.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')] URI: https://mac-v2-signing6.srv.releng.scl3.mozilla.com:9110/token : token generation failed, error message: [('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify failed')]

<type 'exceptions.Exception'>: No more signing servers to try.

<type 'exceptions.Exception'>: No more signing servers to try.

Traceback (most recent call last): Failure: exceptions.Exception: No more signing servers to try. ========= Finished download_token exception (results: 4, elapsed: 45 mins, 14 secs) (at 2016-08-08 18:15:19.691349) ========= ========= Skipped (results: not started, elapsed: not started) ========= ========= Skipped (results: not started, elapsed: not started) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:19.692465) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/fx-team-l64-pgo-00000000000000/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-132.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-l64-pgo-00000000000000/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.005083 ========= master_lag: 0.07 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:19.767359) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:19.767693) ========= rm -f oauth.txt in dir /builds/slave/fx-team-l64-pgo-00000000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-132.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/fx-team-l64-pgo-00000000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002580 ========= master_lag: 0.05 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2016-08-08 18:15:19.817016) ========= ========= Started reboot skipped (results: 3, elapsed: 0 secs) (at 2016-08-08 18:15:19.817316) ========= ========= Finished reboot skipped (results: 3, elapsed: 0 secs) (at 2016-08-08 18:15:19.817596) ========= ========= Total master_lag: 0.31 =========