builder: release-jamun-firefox_bouncer_aliases slave: bld-linux64-spot-039 starttime: 1509477587.29 results: failure (2) revision: a3a6860becd3c9ce755f22a168b2afbab2b7d3d0 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:47.294428) ========= master: http://buildbot-master71.bb.releng.use1.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:47.294858) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:47.295136) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:47.460977) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:47.461303) ========= bash -c 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False --2017-10-31 12:19:47-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py Resolving hg.mozilla.org... 63.245.215.25, 63.245.215.102 Connecting to hg.mozilla.org|63.245.215.25|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12179 (12K) [text/x-python] Saving to: ‘archiver_client.py’ 0K .......... . 100% 187M=0s 2017-10-31 12:19:48 (187 MB/s) - ‘archiver_client.py’ saved [12179/12179] program finished with exit code 0 elapsedTime=0.577957 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:48.050625) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:48.050945) ========= rm -rf scripts properties in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.003323 ========= master_lag: 0.05 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:48.100679) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 3 secs) (at 2017-10-31 12:19:48.101024) ========= bash -c 'python archiver_client.py mozharness --repo projects/jamun --rev a3a6860becd3c9ce755f22a168b2afbab2b7d3d0 --destination scripts --debug' in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', u'python archiver_client.py mozharness --repo projects/jamun --rev a3a6860becd3c9ce755f22a168b2afbab2b7d3d0 --destination scripts --debug'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2017-10-31 12:19:48,151 truncating revision to first 12 chars 2017-10-31 12:19:48,151 Setting DEBUG logging. 2017-10-31 12:19:48,152 attempt 1/10 2017-10-31 12:19:48,152 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/projects/jamun/a3a6860becd3?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2017-10-31 12:19:50,302 unpacking tar archive at: jamun-a3a6860becd3/testing/mozharness/ program finished with exit code 0 elapsedTime=3.386729 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 3 secs) (at 2017-10-31 12:19:51.497893) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:51.498231) ========= script_repo_revision: a3a6860becd3c9ce755f22a168b2afbab2b7d3d0 ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:51.498602) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:51.498933) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2017-10-31 12:19:51.507913) ========= ========= Started '/tools/buildbot/bin/python scripts/scripts/release/postrelease_bouncer_aliases.py ...' failed (results: 2, elapsed: 8 mins, 39 secs) (at 2017-10-31 12:19:51.508183) ========= /tools/buildbot/bin/python scripts/scripts/release/postrelease_bouncer_aliases.py -c releases/bouncer_firefox_beta.py in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/. (timeout 3600 secs) (maxTime 14400 secs) watching logfiles {} argv: ['/tools/buildbot/bin/python', 'scripts/scripts/release/postrelease_bouncer_aliases.py', '-c', 'releases/bouncer_firefox_beta.py'] environment: CCACHE_COMPRESS=1 CCACHE_DIR=/builds/ccache CCACHE_HASHDIR= CCACHE_UMASK=002 DISPLAY=:2 G_BROKEN_FILENAMES=1 HG_SHARE_BASE_DIR=/builds/hg-shared HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LC_ALL=C LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld MOZ_AUTOMATION=1 MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_OBJDIR=obj-firefox PATH=/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin PROPERTIES_FILE=/builds/slave/rel-jamun-fx_bouncer_aliases-0/buildprops.json PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0 SHELL=/bin/bash SHLVL=1 TERM=linux TINDERBOX_OUTPUT=1 TMOUT=86400 TOOLTOOL_CACHE=/builds/tooltool_cache TOOLTOOL_HOME=/builds USER=cltbld _=/tools/buildbot/bin/python using PTY: False 12:19:51 INFO - MultiFileLogger online at 20171031 12:19:51 in /builds/slave/rel-jamun-fx_bouncer_aliases-0 12:19:51 INFO - Using buildbot properties: 12:19:51 INFO - { 12:19:51 INFO - "tuxedo_server_url": "https://admin-bouncer.stage.mozaws.net/api", 12:19:51 INFO - "project": "", 12:19:51 INFO - "product": "firefox", 12:19:51 INFO - "build_number": 27, 12:19:51 INFO - "taskId": "KQNvTM_rTYGRpKFVTyM-fw", 12:19:51 INFO - "repository": "", 12:19:51 INFO - "buildername": "release-jamun-firefox_bouncer_aliases", 12:19:51 INFO - "basedir": "/builds/slave/rel-jamun-fx_bouncer_aliases-0", 12:19:51 INFO - "buildnumber": 1, 12:19:51 INFO - "platform": null, 12:19:51 INFO - "version": "58.0b1", 12:19:51 INFO - "release_promotion": true, 12:19:51 INFO - "script_repo_revision": "a3a6860becd3c9ce755f22a168b2afbab2b7d3d0", 12:19:51 INFO - "branch": "jamun", 12:19:51 INFO - "master": "http://buildbot-master71.bb.releng.use1.mozilla.com:8001/", 12:19:51 INFO - "revision": "a3a6860becd3c9ce755f22a168b2afbab2b7d3d0", 12:19:51 INFO - "slavename": "bld-linux64-spot-039", 12:19:51 INFO - "repo_path": "projects/jamun" 12:19:51 INFO - } 12:19:51 INFO - Overriding tuxedo_server_url with https://admin-bouncer.stage.mozaws.net/api 12:19:51 INFO - Overriding version with 58.0b1 12:19:51 INFO - Run as scripts/scripts/release/postrelease_bouncer_aliases.py -c releases/bouncer_firefox_beta.py 12:19:51 INFO - Dumping config to /builds/slave/rel-jamun-fx_bouncer_aliases-0/logs/localconfig.json. 12:19:51 INFO - {'append_to_log': False, 12:19:51 INFO - 'base_work_dir': '/builds/slave/rel-jamun-fx_bouncer_aliases-0', 12:19:51 INFO - 'buildbot_json_path': 'buildprops.json', 12:19:51 INFO - 'config_files': ('releases/bouncer_firefox_beta.py',), 12:19:51 INFO - 'credentials_file': 'oauth.txt', 12:19:51 INFO - 'log_level': 'info', 12:19:51 INFO - 'log_to_console': True, 12:19:51 INFO - 'opt_config_files': (), 12:19:51 INFO - 'partials': {'releases-dir': {'add-locales': True, 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-i686/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:19:51 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-x86_64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:19:51 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/mac/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:19:51 INFO - 'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win32/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s-Partial-%(prev_version)s', 12:19:51 INFO - 'ssl-only': False}}, 12:19:51 INFO - 'pip_index': True, 12:19:51 INFO - 'products': {'complete-mar': {'add-locales': True, 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-i686/:lang/firefox-%(version)s.complete.mar'}, 12:19:51 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-x86_64/:lang/firefox-%(version)s.complete.mar'}, 12:19:51 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/mac/:lang/firefox-%(version)s.complete.mar'}, 12:19:51 INFO - 'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win32/:lang/firefox-%(version)s.complete.mar'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win64/:lang/firefox-%(version)s.complete.mar'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s-Complete', 12:19:51 INFO - 'ssl-only': False}, 12:19:51 INFO - 'complete-mar-bz2': {'add-locales': True, 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-i686/:lang/firefox-%(version)s.bz2.complete.mar'}, 12:19:51 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/linux-x86_64/:lang/firefox-%(version)s.bz2.complete.mar'}, 12:19:51 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/mac/:lang/firefox-%(version)s.bz2.complete.mar'}, 12:19:51 INFO - 'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win32/:lang/firefox-%(version)s.bz2.complete.mar'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/update/win64/:lang/firefox-%(version)s.bz2.complete.mar'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s-Complete-bz2', 12:19:51 INFO - 'ssl-only': False}, 12:19:51 INFO - 'installer': {'add-locales': True, 12:19:51 INFO - 'alias': 'firefox-beta-latest', 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/linux-i686/:lang/firefox-%(version)s.tar.bz2'}, 12:19:51 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/linux-x86_64/:lang/firefox-%(version)s.tar.bz2'}, 12:19:51 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/mac/:lang/Firefox%%20%(version)s.dmg'}, 12:19:51 INFO - 'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Setup%%20%(version)s.exe'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win64/:lang/Firefox%%20Setup%%20%(version)s.exe'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s', 12:19:51 INFO - 'ssl-only': False}, 12:19:51 INFO - 'installer-ssl': {'add-locales': True, 12:19:51 INFO - 'alias': 'firefox-beta-latest-ssl', 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/linux-i686/:lang/firefox-%(version)s.tar.bz2'}, 12:19:51 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/linux-x86_64/:lang/firefox-%(version)s.tar.bz2'}, 12:19:51 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/mac/:lang/Firefox%%20%(version)s.dmg'}, 12:19:51 INFO - 'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Setup%%20%(version)s.exe'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win64/:lang/Firefox%%20Setup%%20%(version)s.exe'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s-SSL', 12:19:51 INFO - 'ssl-only': True}, 12:19:51 INFO - 'stub-installer': {'add-locales': True, 12:19:51 INFO - 'alias': 'firefox-beta-stub', 12:19:51 INFO - 'check_uptake': True, 12:19:51 INFO - 'paths': {'win32': {'bouncer-platform': 'win', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Installer.exe'}, 12:19:51 INFO - 'win64': {'bouncer-platform': 'win64', 12:19:51 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Installer.exe'}}, 12:19:51 INFO - 'product-name': 'Firefox-%(version)s-stub', 12:19:51 INFO - 'ssl-only': True}}, 12:19:51 INFO - 'shipped-locales-url': 'https://hg.mozilla.org/%(repo)s/raw-file/%(revision)s/browser/locales/shipped-locales', 12:19:51 INFO - 'tuxedo_server_url': u'https://admin-bouncer.stage.mozaws.net/api', 12:19:51 INFO - 'version': u'58.0b1', 12:19:51 INFO - 'virtualenv_modules': ('redo', 'requests'), 12:19:51 INFO - 'virtualenv_path': 'venv', 12:19:51 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 12:19:51 INFO - 'work_dir': 'build'} 12:19:51 INFO - [mozharness: 2017-10-31 19:19:51.673333Z] Running create-virtualenv step. 12:19:51 INFO - Running main action method: create_virtualenv 12:19:51 INFO - Creating virtualenv /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv 12:19:51 INFO - mkdir: /builds/slave/rel-jamun-fx_bouncer_aliases-0/build 12:19:51 INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-jamun-fx_bouncer_aliases-0/scripts/external_tools/virtualenv/virtualenv.py', '--always-copy', '/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv'] in /builds/slave/rel-jamun-fx_bouncer_aliases-0/build 12:19:51 INFO - Copy/paste: /tools/buildbot/bin/python /builds/slave/rel-jamun-fx_bouncer_aliases-0/scripts/external_tools/virtualenv/virtualenv.py --always-copy /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv 12:19:51 INFO - Using partial env: {'VIRTUALENV_NO_DOWNLOAD': '1'} 12:19:52 INFO - Using real prefix '/tools/python27' 12:19:52 INFO - Cannot find file lib (bad symlink) 12:19:52 INFO - New python executable in /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/python 12:19:52 INFO - Overwriting /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/distutils/__init__.py with new content 12:19:53 INFO - Installing setuptools, pip, wheel...done. 12:19:53 INFO - Return code: 0 12:19:53 INFO - Installing redo into virtualenv /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv 12:19:53 INFO - retry: Calling run_command with args: [['/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip', 'install', '--timeout', '120', 'redo']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x2742cb0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7f16ed17d6c0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x2766350>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x271d5e0>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x25f6540>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x2442d78>, 'level': 'critical'}], 'cwd': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/build', 'env': {'MOZ_AUTOMATION': '1', 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 'TMOUT': '86400', 'LOGNAME': 'cltbld', 'USER': 'cltbld', 'MOZ_OBJDIR': 'obj-firefox', 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 'DISPLAY': ':2', 'CCACHE_UMASK': '002', 'LANG': 'en_US.UTF-8', 'CCACHE_HASHDIR': '', 'TERM': 'linux', 'SHELL': '/bin/bash', 'SHLVL': '1', 'G_BROKEN_FILENAMES': '1', 'HISTSIZE': '1000', 'HOME': '/home/cltbld', 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'TINDERBOX_OUTPUT': '1', 'MAIL': '/var/spool/mail/cltbld', 'LC_ALL': 'C', 'TOOLTOOL_HOME': '/builds', '_': '/tools/buildbot/bin/python', 'HOSTNAME': 'bld-linux64-spot-039.build.releng.use1.mozilla.com', 'HISTCONTROL': 'ignoredups', 'PWD': '/builds/slave/rel-jamun-fx_bouncer_aliases-0', 'PROPERTIES_FILE': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/buildprops.json', 'CCACHE_DIR': '/builds/ccache', 'CCACHE_COMPRESS': '1', 'TOOLTOOL_CACHE': '/builds/tooltool_cache'}}, attempt #1 12:19:53 INFO - Running command: ['/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip', 'install', '--timeout', '120', 'redo'] in /builds/slave/rel-jamun-fx_bouncer_aliases-0/build 12:19:53 INFO - Copy/paste: /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip install --timeout 120 redo 12:19:53 INFO - Using env: {'CCACHE_COMPRESS': '1', 12:19:53 INFO - 'CCACHE_DIR': '/builds/ccache', 12:19:53 INFO - 'CCACHE_HASHDIR': '', 12:19:53 INFO - 'CCACHE_UMASK': '002', 12:19:53 INFO - 'DISPLAY': ':2', 12:19:53 INFO - 'G_BROKEN_FILENAMES': '1', 12:19:53 INFO - 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 12:19:53 INFO - 'HISTCONTROL': 'ignoredups', 12:19:53 INFO - 'HISTSIZE': '1000', 12:19:53 INFO - 'HOME': '/home/cltbld', 12:19:53 INFO - 'HOSTNAME': 'bld-linux64-spot-039.build.releng.use1.mozilla.com', 12:19:53 INFO - 'LANG': 'en_US.UTF-8', 12:19:53 INFO - 'LC_ALL': 'C', 12:19:53 INFO - 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 12:19:53 INFO - 'LOGNAME': 'cltbld', 12:19:53 INFO - 'MAIL': '/var/spool/mail/cltbld', 12:19:53 INFO - 'MOZ_AUTOMATION': '1', 12:19:53 INFO - 'MOZ_CRASHREPORTER_NO_REPORT': '1', 12:19:53 INFO - 'MOZ_OBJDIR': 'obj-firefox', 12:19:53 INFO - 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 12:19:53 INFO - 'PROPERTIES_FILE': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/buildprops.json', 12:19:53 INFO - 'PWD': '/builds/slave/rel-jamun-fx_bouncer_aliases-0', 12:19:53 INFO - 'SHELL': '/bin/bash', 12:19:53 INFO - 'SHLVL': '1', 12:19:53 INFO - 'TERM': 'linux', 12:19:53 INFO - 'TINDERBOX_OUTPUT': '1', 12:19:53 INFO - 'TMOUT': '86400', 12:19:53 INFO - 'TOOLTOOL_CACHE': '/builds/tooltool_cache', 12:19:53 INFO - 'TOOLTOOL_HOME': '/builds', 12:19:53 INFO - 'USER': 'cltbld', 12:19:53 INFO - '_': '/tools/buildbot/bin/python'} 12:19:54 INFO - Ignoring indexes: https://pypi.python.org/simple 12:19:54 INFO - Collecting redo 12:19:56 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/redo-1.5-py2.py3-none-any.whl 12:19:56 INFO - Installing collected packages: redo 12:19:56 INFO - Successfully installed redo-1.5 12:19:56 INFO - Return code: 0 12:19:56 INFO - Installing requests into virtualenv /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv 12:19:56 INFO - retry: Calling run_command with args: [['/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip', 'install', '--timeout', '120', 'requests']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x2742cb0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x7f16ed17d6c0>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x2766350>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x271d5e0>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x25f6540>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x2442d78>, 'level': 'critical'}], 'cwd': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/build', 'env': {'MOZ_AUTOMATION': '1', 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 'TMOUT': '86400', 'LOGNAME': 'cltbld', 'USER': 'cltbld', 'MOZ_OBJDIR': 'obj-firefox', 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 'DISPLAY': ':2', 'CCACHE_UMASK': '002', 'LANG': 'en_US.UTF-8', 'CCACHE_HASHDIR': '', 'TERM': 'linux', 'SHELL': '/bin/bash', 'SHLVL': '1', 'G_BROKEN_FILENAMES': '1', 'HISTSIZE': '1000', 'HOME': '/home/cltbld', 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'TINDERBOX_OUTPUT': '1', 'MAIL': '/var/spool/mail/cltbld', 'LC_ALL': 'C', 'TOOLTOOL_HOME': '/builds', '_': '/tools/buildbot/bin/python', 'HOSTNAME': 'bld-linux64-spot-039.build.releng.use1.mozilla.com', 'HISTCONTROL': 'ignoredups', 'PWD': '/builds/slave/rel-jamun-fx_bouncer_aliases-0', 'PROPERTIES_FILE': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/buildprops.json', 'CCACHE_DIR': '/builds/ccache', 'CCACHE_COMPRESS': '1', 'TOOLTOOL_CACHE': '/builds/tooltool_cache'}}, attempt #1 12:19:56 INFO - Running command: ['/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip', 'install', '--timeout', '120', 'requests'] in /builds/slave/rel-jamun-fx_bouncer_aliases-0/build 12:19:56 INFO - Copy/paste: /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip install --timeout 120 requests 12:19:56 INFO - Using env: {'CCACHE_COMPRESS': '1', 12:19:56 INFO - 'CCACHE_DIR': '/builds/ccache', 12:19:56 INFO - 'CCACHE_HASHDIR': '', 12:19:56 INFO - 'CCACHE_UMASK': '002', 12:19:56 INFO - 'DISPLAY': ':2', 12:19:56 INFO - 'G_BROKEN_FILENAMES': '1', 12:19:56 INFO - 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 12:19:56 INFO - 'HISTCONTROL': 'ignoredups', 12:19:56 INFO - 'HISTSIZE': '1000', 12:19:56 INFO - 'HOME': '/home/cltbld', 12:19:56 INFO - 'HOSTNAME': 'bld-linux64-spot-039.build.releng.use1.mozilla.com', 12:19:56 INFO - 'LANG': 'en_US.UTF-8', 12:19:56 INFO - 'LC_ALL': 'C', 12:19:56 INFO - 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 12:19:56 INFO - 'LOGNAME': 'cltbld', 12:19:56 INFO - 'MAIL': '/var/spool/mail/cltbld', 12:19:56 INFO - 'MOZ_AUTOMATION': '1', 12:19:56 INFO - 'MOZ_CRASHREPORTER_NO_REPORT': '1', 12:19:56 INFO - 'MOZ_OBJDIR': 'obj-firefox', 12:19:56 INFO - 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 12:19:56 INFO - 'PROPERTIES_FILE': '/builds/slave/rel-jamun-fx_bouncer_aliases-0/buildprops.json', 12:19:56 INFO - 'PWD': '/builds/slave/rel-jamun-fx_bouncer_aliases-0', 12:19:56 INFO - 'SHELL': '/bin/bash', 12:19:56 INFO - 'SHLVL': '1', 12:19:56 INFO - 'TERM': 'linux', 12:19:56 INFO - 'TINDERBOX_OUTPUT': '1', 12:19:56 INFO - 'TMOUT': '86400', 12:19:56 INFO - 'TOOLTOOL_CACHE': '/builds/tooltool_cache', 12:19:56 INFO - 'TOOLTOOL_HOME': '/builds', 12:19:56 INFO - 'USER': 'cltbld', 12:19:56 INFO - '_': '/tools/buildbot/bin/python'} 12:19:57 INFO - Ignoring indexes: https://pypi.python.org/simple 12:19:57 INFO - Collecting requests 12:19:59 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/requests-2.18.3-py2.py3-none-any.whl (88kB) 12:20:00 INFO - Collecting certifi>=2017.4.17 (from requests) 12:20:02 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/certifi-2017.7.27.1-py2.py3-none-any.whl (349kB) 12:20:03 INFO - Collecting chardet<3.1.0,>=3.0.2 (from requests) 12:20:05 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/chardet-3.0.4-py2.py3-none-any.whl (133kB) 12:20:05 INFO - Collecting urllib3<1.23,>=1.21.1 (from requests) 12:20:08 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/urllib3-1.22-py2.py3-none-any.whl (132kB) 12:20:08 INFO - Collecting idna<2.6,>=2.5 (from requests) 12:20:11 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/idna-2.5-py2.py3-none-any.whl (55kB) 12:20:11 INFO - Installing collected packages: certifi, chardet, urllib3, idna, requests 12:20:11 INFO - Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.5 requests-2.18.3 urllib3-1.22 12:20:11 INFO - Return code: 0 12:20:11 INFO - Done creating virtualenv /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv. 12:20:11 INFO - Getting output from command: ['/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip', 'freeze'] 12:20:11 INFO - Copy/paste: /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/bin/pip freeze 12:20:12 INFO - Reading from file tmpfile_stdout 12:20:12 INFO - Reading from file tmpfile_stderr 12:20:12 INFO - Current package versions: 12:20:12 INFO - certifi == 2017.7.27.1 12:20:12 INFO - chardet == 3.0.4 12:20:12 INFO - idna == 2.5 12:20:12 INFO - redo == 1.5 12:20:12 INFO - requests == 2.18.3 12:20:12 INFO - urllib3 == 1.22 12:20:12 INFO - [mozharness: 2017-10-31 19:20:12.727352Z] Finished create-virtualenv step (success) 12:20:12 INFO - [mozharness: 2017-10-31 19:20:12.727554Z] Running activate-virtualenv step. 12:20:12 INFO - Running main action method: activate_virtualenv 12:20:12 INFO - [mozharness: 2017-10-31 19:20:12.728780Z] Finished activate-virtualenv step (success) 12:20:12 INFO - [mozharness: 2017-10-31 19:20:12.729013Z] Running update-bouncer-aliases step. 12:20:12 INFO - Running main action method: update_bouncer_aliases 12:20:12 INFO - Updating firefox-beta-latest to point to Firefox-58.0b1 using https://admin-bouncer.stage.mozaws.net/api/create_update_alias /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings SNIMissingWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 12:21:13 INFO - retry: Calling with args: (), kwargs: {}, attempt #2 /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 12:22:50 INFO - retry: Calling with args: (), kwargs: {}, attempt #3 /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 12:25:07 INFO - retry: Calling with args: (), kwargs: {}, attempt #4 /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 12:28:29 INFO - retry: Calling with args: (), kwargs: {}, attempt #5 /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecurePlatformWarning /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/urllib3/connectionpool.py:858: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) 12:28:30 INFO - retry: Giving up on 12:28:30 INFO - [mozharness: 2017-10-31 19:28:30.519446Z] Finished update-bouncer-aliases step (failed) 12:28:30 FATAL - Uncaught exception: Traceback (most recent call last): 12:28:30 FATAL - File "/builds/slave/rel-jamun-fx_bouncer_aliases-0/scripts/mozharness/base/script.py", line 2059, in run 12:28:30 FATAL - self.run_action(action) 12:28:30 FATAL - File "/builds/slave/rel-jamun-fx_bouncer_aliases-0/scripts/mozharness/base/script.py", line 1998, in run_action 12:28:30 FATAL - self._possibly_run_method(method_name, error_if_missing=True) 12:28:30 FATAL - File "/builds/slave/rel-jamun-fx_bouncer_aliases-0/scripts/mozharness/base/script.py", line 1938, in _possibly_run_method 12:28:30 FATAL - return getattr(self, method_name)() 12:28:30 FATAL - File "scripts/scripts/release/postrelease_bouncer_aliases.py", line 102, in update_bouncer_aliases 12:28:30 FATAL - related_product, info["alias"]) 12:28:30 FATAL - File "scripts/scripts/release/postrelease_bouncer_aliases.py", line 87, in _update_bouncer_alias 12:28:30 FATAL - retry(do_update_bouncer_alias) 12:28:30 FATAL - File "/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/redo/__init__.py", line 152, in retry 12:28:30 FATAL - return action(*args, **kwargs) 12:28:30 FATAL - File "scripts/scripts/release/postrelease_bouncer_aliases.py", line 85, in do_update_bouncer_alias 12:28:30 FATAL - r.raise_for_status() 12:28:30 FATAL - File "/builds/slave/rel-jamun-fx_bouncer_aliases-0/build/venv/lib/python2.7/site-packages/requests/models.py", line 935, in raise_for_status 12:28:30 FATAL - raise HTTPError(http_error_msg, response=self) 12:28:30 FATAL - HTTPError: 400 Client Error: BAD REQUEST for url: https://admin-bouncer.stage.mozaws.net/api/create_update_alias 12:28:30 FATAL - Running post_fatal callback... 12:28:30 FATAL - Exiting -1 12:28:30 INFO - Running post-run listener: copy_logs_to_upload_dir 12:28:30 INFO - Copying logs to upload dir... 12:28:30 INFO - mkdir: /builds/slave/rel-jamun-fx_bouncer_aliases-0/build/upload/logs program finished with exit code 255 elapsedTime=519.028327 ========= master_lag: 0.01 ========= ========= Finished '/tools/buildbot/bin/python scripts/scripts/release/postrelease_bouncer_aliases.py ...' failed (results: 2, elapsed: 8 mins, 39 secs) (at 2017-10-31 12:28:30.548897) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2017-10-31 12:28:30.551382) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.005030 ========= master_lag: 0.05 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2017-10-31 12:28:30.605117) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:28:30.605461) ========= rm -f oauth.txt in dir /builds/slave/rel-jamun-fx_bouncer_aliases-0/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-039.build.releng.use1.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-jamun-fx_bouncer_aliases-0 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002499 ========= master_lag: 0.05 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2017-10-31 12:28:30.653702) ========= ========= Started reboot skipped (results: 3, elapsed: 0 secs) (at 2017-10-31 12:28:30.654012) ========= ========= Finished reboot skipped (results: 3, elapsed: 0 secs) (at 2017-10-31 12:28:30.654293) ========= ========= Total master_lag: 0.17 =========