builder: mozilla-aurora_win7-ix_test_pgo-mochitest-e10s-devtools-chrome-1 slave: t-w732-ix-194 starttime: 1461191332.51 results: exception (4) buildid: 20160420120429 builduid: ae4579530dd54e17a53496a45dc9bbcc revision: 5649dee23169c659655bfd2f9bae78b5b1bed073 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2016-04-20 15:28:52.514785) ========= master: http://buildbot-master127.bb.releng.scl3.mozilla.com:8201/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2016-04-20 15:28:52.515219) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-04-20 15:28:52.515497) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2016-04-20 15:28:52.593125) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2016-04-20 15:28:52.593833) ========= 'bash' '-c' 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir C:\\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-194 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-194 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;c:\mozilla-build\hg\;c:\Program Files\Microsoft Windows Performance Toolkit\;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-194 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log using PTY: False --15:28:53-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py => `archiver_client.py' Resolving hg.mozilla.org... 63.245.215.102, 63.245.215.25 Connecting to hg.mozilla.org|63.245.215.102|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12,141 (12K) [text/x-python] 0K .......... . 100% 32.53 MB/s 15:28:54 (32.53 MB/s) - `archiver_client.py' saved [12141/12141] program finished with exit code 0 elapsedTime=1.433000 ========= master_lag: 0.08 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2016-04-20 15:28:54.102577) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 5 secs) (at 2016-04-20 15:28:54.102927) ========= 'rm' '-rf' 'scripts' 'properties' in dir C:\\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-194 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-194 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;c:\mozilla-build\hg\;c:\Program Files\Microsoft Windows Performance Toolkit\;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-194 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log using PTY: False program finished with exit code 0 elapsedTime=4.101000 ========= master_lag: 1.14 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 5 secs) (at 2016-04-20 15:28:59.348899) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 9 secs) (at 2016-04-20 15:28:59.349286) ========= 'bash' '-c' u'python archiver_client.py mozharness --repo releases/mozilla-aurora --rev 5649dee23169c659655bfd2f9bae78b5b1bed073 --destination scripts --debug' in dir C:\\slave\test\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', u'python archiver_client.py mozharness --repo releases/mozilla-aurora --rev 5649dee23169c659655bfd2f9bae78b5b1bed073 --destination scripts --debug'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-194 COMSPEC=C:\windows\system32\cmd.exe DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-194 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Program Files\NVIDIA Corporation\PhysX\Common;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\msys\bin;C:\mozilla-build\vim\vim72;C:\mozilla-build\wget;C:\mozilla-build\info-zip;C:\CoreUtils\bin;C:\mozilla-build\buildbotve\scripts;c:\mozilla-build\hg\;c:\Program Files\Microsoft Windows Performance Toolkit\;C:\mozilla-build\hg PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-194 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log using PTY: False 2016-04-20 15:29:00,742 truncating revision to first 12 chars 2016-04-20 15:29:00,742 Setting DEBUG logging. 2016-04-20 15:29:00,742 attempt 1/10 2016-04-20 15:29:00,742 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/releases/mozilla-aurora/5649dee23169?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2016-04-20 15:29:01,680 unpacking tar archive at: mozilla-aurora-5649dee23169/testing/mozharness/ program finished with exit code 0 elapsedTime=3.076000 ========= master_lag: 6.34 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 9 secs) (at 2016-04-20 15:29:08.769183) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-04-20 15:29:08.769511) ========= script_repo_revision: 5649dee23169c659655bfd2f9bae78b5b1bed073 ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2016-04-20 15:29:08.769914) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-04-20 15:29:08.770196) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2016-04-20 15:29:09.157065) ========= ========= Started run_script exception (results: 4, elapsed: 16 hrs, 15 mins, 12 secs) (at 2016-04-20 15:29:09.157443) ========= remoteFailed: [Failure instance: Traceback (failure with no frames): : [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion. ] ]

<class 'twisted.spread.pb.PBConnectionLost'>: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion. ]

<class 'twisted.spread.pb.PBConnectionLost'>: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion. ]

Traceback (most recent call last): Failure: twisted.spread.pb.PBConnectionLost: [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion. ] ========= Finished run_script exception (results: 4, elapsed: 16 hrs, 15 mins, 12 secs) (at 2016-04-21 07:44:22.088291) ========= ========= Started set_script_properties exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.088733) ========= remoteFailed: [Failure instance: Traceback: : Calling Stale Broker /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py:225:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1071:startCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1002:runCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:77:run --- --- /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py:125:maybeDeferred /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:529:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:311:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:112:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:328:callRemote /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:807:_sendMessage ]

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py, line 125 in maybeDeferred
123  """
124  try:
125    result = f(*args, **kw)
126  except:
Locals
args'()'
kw'{}'
f"<bound method RemoteShellCommand.start of <RemoteShellCommand '['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f']'>>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 529 in start
527                       self.args['workdir'])
528    log.msg(what)
529    return LoggedRemoteCommand.start(self)
530
Locals
what'"command \'[\'cmd\', \'/C\', \'for\', \'%f\', \'in\', \'(*)\', \'do\', \'@type\', \'%f\']\' in dir \'properties\'"'
self"<RemoteShellCommand '['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f']'>"
Globals
LoggedRemoteCommand'<class buildbot.process.buildstep.LoggedRemoteCommand at 0x2773940>'
log"<module 'twisted.python.log' from '/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/python/log.pyc'>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 311 in start
309          % self)
310    self.updates = {}
311    return RemoteCommand.start(self)
312
Locals
self"<RemoteShellCommand '['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f']'>"
Globals
RemoteCommand'<class buildbot.process.buildstep.RemoteCommand at 0x27733f8>'
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 112 in start
110    # We should fire self.deferred when the command is done.
111    d = self.remote.callRemote("startCommand", self, self.commandID,
112                  self.remote_command, cmd_args)
113    return d
Locals
self"<RemoteShellCommand '['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f']'>"
cmd_args"{'workdir': 'properties', 'command': ['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f'], 'timeout': 1200, 'env': None, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}}"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 328 in callRemote
326
327    return self.broker._sendMessage('',self.perspective, self.luid,
328                    _name, args, kw)
329
Locals
self'<twisted.spread.pb.RemoteReference instance at 0x87517518>'
args"(<RemoteShellCommand '['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f']'>, '427020', 'shell', {'workdir': 'properties', 'command': ['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f'], 'timeout': 1200, 'env': None, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}})"
kw'{}'
_name"'startCommand'"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 807 in _sendMessage
805      del kw['pbanswer']
806    if self.disconnected:
807      raise DeadReferenceError("Calling Stale Broker")
808    try:
Locals
self'<twisted.spread.pb.Broker instance at 0x78a30998>'
kw'{}'
Globals
Broker'<class twisted.spread.pb.Broker at 0x2359120>'
DeadReferenceError"<class 'twisted.spread.pb.DeadReferenceError'>"

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

Traceback (most recent call last): File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py", line 225, in start self.startCommand(cmd, warnings) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1071, in startCommand d = self.runCommand(cmd) # might raise ConnectionLost File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1002, in runCommand d = c.run(self, self.remote) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 77, in run d = defer.maybeDeferred(self.start) --- --- File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py", line 125, in maybeDeferred result = f(*args, **kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 529, in start return LoggedRemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 311, in start return RemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 112, in start self.remote_command, cmd_args) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 328, in callRemote _name, args, kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 807, in _sendMessage raise DeadReferenceError("Calling Stale Broker") twisted.spread.pb.DeadReferenceError: Calling Stale Broker ========= Finished set_script_properties exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.237281) ========= ========= Started rm_oauth_txt exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.238266) ========= remoteFailed: [Failure instance: Traceback: : Calling Stale Broker /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py:225:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1071:startCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1002:runCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:77:run --- --- /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py:125:maybeDeferred /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:529:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:311:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:112:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:328:callRemote /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:807:_sendMessage ]

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py, line 125 in maybeDeferred
123  """
124  try:
125    result = f(*args, **kw)
126  except:
Locals
args'()'
kw'{}'
f"<bound method RemoteShellCommand.start of <RemoteShellCommand '['rm', '-f', 'oauth.txt']'>>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 529 in start
527                       self.args['workdir'])
528    log.msg(what)
529    return LoggedRemoteCommand.start(self)
530
Locals
what'"command \'[\'rm\', \'-f\', \'oauth.txt\']\' in dir \'.\'"'
self"<RemoteShellCommand '['rm', '-f', 'oauth.txt']'>"
Globals
LoggedRemoteCommand'<class buildbot.process.buildstep.LoggedRemoteCommand at 0x2773940>'
log"<module 'twisted.python.log' from '/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/python/log.pyc'>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 311 in start
309          % self)
310    self.updates = {}
311    return RemoteCommand.start(self)
312
Locals
self"<RemoteShellCommand '['rm', '-f', 'oauth.txt']'>"
Globals
RemoteCommand'<class buildbot.process.buildstep.RemoteCommand at 0x27733f8>'
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 112 in start
110    # We should fire self.deferred when the command is done.
111    d = self.remote.callRemote("startCommand", self, self.commandID,
112                  self.remote_command, cmd_args)
113    return d
Locals
self"<RemoteShellCommand '['rm', '-f', 'oauth.txt']'>"
cmd_args"{'workdir': '.', 'command': ['rm', '-f', 'oauth.txt'], 'timeout': 1200, 'env': None, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}}"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 328 in callRemote
326
327    return self.broker._sendMessage('',self.perspective, self.luid,
328                    _name, args, kw)
329
Locals
self'<twisted.spread.pb.RemoteReference instance at 0x87517518>'
args"(<RemoteShellCommand '['rm', '-f', 'oauth.txt']'>, '427021', 'shell', {'workdir': '.', 'command': ['rm', '-f', 'oauth.txt'], 'timeout': 1200, 'env': None, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}})"
kw'{}'
_name"'startCommand'"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 807 in _sendMessage
805      del kw['pbanswer']
806    if self.disconnected:
807      raise DeadReferenceError("Calling Stale Broker")
808    try:
Locals
self'<twisted.spread.pb.Broker instance at 0x78a30998>'
kw'{}'
Globals
Broker'<class twisted.spread.pb.Broker at 0x2359120>'
DeadReferenceError"<class 'twisted.spread.pb.DeadReferenceError'>"

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

Traceback (most recent call last): File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py", line 225, in start self.startCommand(cmd, warnings) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1071, in startCommand d = self.runCommand(cmd) # might raise ConnectionLost File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1002, in runCommand d = c.run(self, self.remote) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 77, in run d = defer.maybeDeferred(self.start) --- --- File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py", line 125, in maybeDeferred result = f(*args, **kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 529, in start return LoggedRemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 311, in start return RemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 112, in start self.remote_command, cmd_args) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 328, in callRemote _name, args, kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 807, in _sendMessage raise DeadReferenceError("Calling Stale Broker") twisted.spread.pb.DeadReferenceError: Calling Stale Broker ========= Finished rm_oauth_txt exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.267269) ========= ========= Started reboot exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.268348) ========= remoteFailed: [Failure instance: Traceback: : Calling Stale Broker /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py:225:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1071:startCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:1002:runCommand /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:77:run --- --- /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py:125:maybeDeferred /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:529:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:311:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py:112:start /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:328:callRemote /builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py:807:_sendMessage ]

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py, line 125 in maybeDeferred
123  """
124  try:
125    result = f(*args, **kw)
126  except:
Locals
args'()'
kw'{}'
f"<bound method RemoteShellCommand.start of <RemoteShellCommand '['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z']'>>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 529 in start
527                       self.args['workdir'])
528    log.msg(what)
529    return LoggedRemoteCommand.start(self)
530
Locals
what'"command \'[\'c:/mozilla-build/python27/python\', \'-u\', \'scripts/external_tools/count_and_reboot.py\', \'-f\', \'../reboot_count.txt\', \'-n\', \'1\', \'-z\']\' in dir \'.\'"'
self"<RemoteShellCommand '['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z']'>"
Globals
LoggedRemoteCommand'<class buildbot.process.buildstep.LoggedRemoteCommand at 0x2773940>'
log"<module 'twisted.python.log' from '/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/python/log.pyc'>"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 311 in start
309          % self)
310    self.updates = {}
311    return RemoteCommand.start(self)
312
Locals
self"<RemoteShellCommand '['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z']'>"
Globals
RemoteCommand'<class buildbot.process.buildstep.RemoteCommand at 0x27733f8>'
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py, line 112 in start
110    # We should fire self.deferred when the command is done.
111    d = self.remote.callRemote("startCommand", self, self.commandID,
112                  self.remote_command, cmd_args)
113    return d
Locals
self"<RemoteShellCommand '['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z']'>"
cmd_args"{'workdir': '.', 'command': ['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z'], 'timeout': 1200, 'env': {'MOZILLABUILD': 'D:\\\\mozilla-build', 'MOZ_TOOLS': 'D:\\\\mozilla-build\\\\moztools', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'MOZ_NO_REMOTE': '1', 'MOZ_HIDE_RESULTS_TABLE': '1', 'LIBPATH': 'C:\\\\WINDOWS\\\\Microsoft.NET\\\\Framework\\\\v2.0.50727;D:\\\\msvs8\\\\VC\\\\ATLMFC\\\\LIB', 'MOZILLABUILDPATH': '\\\\mozilla-build\\\\', 'NO_FAIL_ON_TEST_ERRORS': '1', 'NO_EM_RESTART': '1', 'MSVCDir': 'D:\\\\msvs8\\\\VC', 'XPCOM_DEBUG_BREAK': 'warn', 'MOZ_AIRBAG': '1', 'MOZ_MSVCVERSION': '8', 'VCVARS': 'D:\\\\msvs8\\\\VC\\\\bin\\\\vcvars32.bat', 'VSINSTALLDIR': 'D:\\\\msvs8', 'DevEnvDir': 'D:\\\\msvs8\\\\Common7\\\\IDE', 'MOZILLABUILDDRIVE': 'C:', 'PROPERTIES_FILE': 'C:\\\\slave\\\\test/buildprops.json', 'FrameworkDir': 'C:\\\\WINDOWS\\\\Microsoft.NET\\\\Framework', 'FrameworkVersion': 'v2.0.50727', 'FrameworkSDKDir': 'D:\\\\msvs8\\\\SDK\\\\v2.0', 'VCINSTALLDIR': 'D:\\\\msvs8\\\\VC'}, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}}"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 328 in callRemote
326
327    return self.broker._sendMessage('',self.perspective, self.luid,
328                    _name, args, kw)
329
Locals
self'<twisted.spread.pb.RemoteReference instance at 0x87517518>'
args"(<RemoteShellCommand '['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z']'>, '427022', 'shell', {'workdir': '.', 'command': ['c:/mozilla-build/python27/python', '-u', 'scripts/external_tools/count_and_reboot.py', '-f', '../reboot_count.txt', '-n', '1', '-z'], 'timeout': 1200, 'env': {'MOZILLABUILD': 'D:\\\\mozilla-build', 'MOZ_TOOLS': 'D:\\\\mozilla-build\\\\moztools', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'MOZ_NO_REMOTE': '1', 'MOZ_HIDE_RESULTS_TABLE': '1', 'LIBPATH': 'C:\\\\WINDOWS\\\\Microsoft.NET\\\\Framework\\\\v2.0.50727;D:\\\\msvs8\\\\VC\\\\ATLMFC\\\\LIB', 'MOZILLABUILDPATH': '\\\\mozilla-build\\\\', 'NO_FAIL_ON_TEST_ERRORS': '1', 'NO_EM_RESTART': '1', 'MSVCDir': 'D:\\\\msvs8\\\\VC', 'XPCOM_DEBUG_BREAK': 'warn', 'MOZ_AIRBAG': '1', 'MOZ_MSVCVERSION': '8', 'VCVARS': 'D:\\\\msvs8\\\\VC\\\\bin\\\\vcvars32.bat', 'VSINSTALLDIR': 'D:\\\\msvs8', 'DevEnvDir': 'D:\\\\msvs8\\\\Common7\\\\IDE', 'MOZILLABUILDDRIVE': 'C:', 'PROPERTIES_FILE': 'C:\\\\slave\\\\test/buildprops.json', 'FrameworkDir': 'C:\\\\WINDOWS\\\\Microsoft.NET\\\\Framework', 'FrameworkVersion': 'v2.0.50727', 'FrameworkSDKDir': 'D:\\\\msvs8\\\\SDK\\\\v2.0', 'VCINSTALLDIR': 'D:\\\\msvs8\\\\VC'}, 'want_stdout': 1, 'usePTY': 'slave-config', 'maxTime': None, 'logEnviron': True, 'want_stderr': 1, 'logfiles': {}})"
kw'{}'
_name"'startCommand'"
/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py, line 807 in _sendMessage
805      del kw['pbanswer']
806    if self.disconnected:
807      raise DeadReferenceError("Calling Stale Broker")
808    try:
Locals
self'<twisted.spread.pb.Broker instance at 0x78a30998>'
kw'{}'
Globals
Broker'<class twisted.spread.pb.Broker at 0x2359120>'
DeadReferenceError"<class 'twisted.spread.pb.DeadReferenceError'>"

<class 'twisted.spread.pb.DeadReferenceError'>: Calling Stale Broker

Traceback (most recent call last): File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/steps/shell.py", line 225, in start self.startCommand(cmd, warnings) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1071, in startCommand d = self.runCommand(cmd) # might raise ConnectionLost File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 1002, in runCommand d = c.run(self, self.remote) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 77, in run d = defer.maybeDeferred(self.start) --- --- File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/internet/defer.py", line 125, in maybeDeferred result = f(*args, **kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 529, in start return LoggedRemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 311, in start return RemoteCommand.start(self) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/buildbot-0.8.2_hg_8b87b4974e3c_production_0.8-py2.7.egg/buildbot/process/buildstep.py", line 112, in start self.remote_command, cmd_args) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 328, in callRemote _name, args, kw) File "/builds/buildbot/tests1-windows/lib/python2.7/site-packages/twisted/spread/pb.py", line 807, in _sendMessage raise DeadReferenceError("Calling Stale Broker") twisted.spread.pb.DeadReferenceError: Calling Stale Broker ========= Finished reboot exception (results: 4, elapsed: 0 secs) (at 2016-04-21 07:44:22.297036) ========= ========= Total master_lag: 7.56 =========