KernelCI setup in LF IT

I think this is done.

The kernelci-docker.automotivelinux.org site now has a successful mongo restore that adds data to the Boots section of the UI.

Here is the output of the seed.sh:

 # ./seed.sh ./kernelci_db_dump.tar.gz
+ DUMP_FILE=./kernelci_db_dump.tar.gz
+ '[' '!' -e ./kernelci_db_dump.tar.gz ']'
++ docker ps -q --filter label=com.docker.swarm.service.name=kernelci_mongo
+ ID=888aad3545a1
+ '[' 888aad3545a1 = '' ']'
+ tar xvf ./kernelci_db_dump.tar.gz
kernelci_db_dump/
kernelci_db_dump/kernel-ci/
kernelci_db_dump/kernel-ci/system.indexes.bson
kernelci_db_dump/kernel-ci/job.bson
kernelci_db_dump/kernel-ci/job.metadata.json
kernelci_db_dump/kernel-ci/boot.bson
kernelci_db_dump/kernel-ci/boot.metadata.json
kernelci_db_dump/kernel-ci/build.bson
kernelci_db_dump/kernel-ci/build.metadata.json
kernelci_db_dump/kernel-ci/api-token.bson
kernelci_db_dump/kernel-ci/api-token.metadata.json
kernelci_db_dump/kernel-ci/lab.bson
kernelci_db_dump/kernel-ci/lab.metadata.json
kernelci_db_dump/kernel-ci/bisect.bson
kernelci_db_dump/kernel-ci/bisect.metadata.json
kernelci_db_dump/kernel-ci/error_logs.bson
kernelci_db_dump/kernel-ci/error_logs.metadata.json
kernelci_db_dump/kernel-ci/daily_stats.bson
kernelci_db_dump/kernel-ci/daily_stats.metadata.json
kernelci_db_dump/kernel-ci/report.bson
kernelci_db_dump/kernel-ci/report.metadata.json
kernelci_db_dump/kernel-ci/test_suite.bson
kernelci_db_dump/kernel-ci/test_suite.metadata.json
kernelci_db_dump/kernel-ci/test_set.bson
kernelci_db_dump/kernel-ci/test_set.metadata.json
kernelci_db_dump/kernel-ci/test_case.bson
kernelci_db_dump/kernel-ci/test_case.metadata.json
kernelci_db_dump/kernel-ci/boot_regressions.bson
kernelci_db_dump/kernel-ci/boot_regressions.metadata.json
kernelci_db_dump/kernel-ci/boot_regressions_by_boot_id.bson
kernelci_db_dump/kernel-ci/boot_regressions_by_boot_id.metadata.json
+ '[' 0 -eq 0 ']'
+ echo '--Databse backup extracted correctly--'
--Databse backup extracted correctly--
++ sed -e s/.tar.gz//
+ DUMP_FOLDER=./kernelci_db_dump
+ docker cp ./kernelci_db_dump 888aad3545a1:/tmp
+ rm -r ././kernelci_db_dump
+ docker exec 888aad3545a1 /bin/bash -c 'mongorestore --db=kernel-ci /tmp/./kernelci_db_dump/kernel-ci'
2018-01-03T22:07:00.491+0000 the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2018-01-03T22:07:00.491+0000 building a list of collections to restore from /tmp/kernelci_db_dump/kernel-ci dir
2018-01-03T22:07:00.492+0000 reading metadata for kernel-ci.test_case from /tmp/kernelci_db_dump/kernel-ci/test_case.metadata.json
2018-01-03T22:07:00.492+0000 reading metadata for kernel-ci.boot_regressions from /tmp/kernelci_db_dump/kernel-ci/boot_regressions.metadata.json
2018-01-03T22:07:00.493+0000 reading metadata for kernel-ci.test_suite from /tmp/kernelci_db_dump/kernel-ci/test_suite.metadata.json
2018-01-03T22:07:00.493+0000 reading metadata for kernel-ci.test_set from /tmp/kernelci_db_dump/kernel-ci/test_set.metadata.json
2018-01-03T22:07:00.503+0000 restoring kernel-ci.test_case from /tmp/kernelci_db_dump/kernel-ci/test_case.bson
2018-01-03T22:07:00.513+0000 restoring kernel-ci.test_set from /tmp/kernelci_db_dump/kernel-ci/test_set.bson
2018-01-03T22:07:00.523+0000 restoring kernel-ci.test_suite from /tmp/kernelci_db_dump/kernel-ci/test_suite.bson
2018-01-03T22:07:00.543+0000 restoring kernel-ci.boot_regressions from /tmp/kernelci_db_dump/kernel-ci/boot_regressions.bson
2018-01-03T22:07:00.635+0000 no indexes to restore
2018-01-03T22:07:00.635+0000 finished restoring kernel-ci.test_suite (4231 documents)
2018-01-03T22:07:00.635+0000 reading metadata for kernel-ci.boot from /tmp/kernelci_db_dump/kernel-ci/boot.metadata.json
2018-01-03T22:07:00.635+0000 restoring kernel-ci.boot from /tmp/kernelci_db_dump/kernel-ci/boot.bson
2018-01-03T22:07:00.656+0000 no indexes to restore
2018-01-03T22:07:00.656+0000 finished restoring kernel-ci.test_set (4808 documents)
2018-01-03T22:07:00.656+0000 reading metadata for kernel-ci.boot_regressions_by_boot_id from /tmp/kernelci_db_dump/kernel-ci/boot_regressions_by_boot_id.metadata.json
2018-01-03T22:07:00.676+0000 restoring kernel-ci.boot_regressions_by_boot_id from /tmp/kernelci_db_dump/kernel-ci/boot_regressions_by_boot_id.bson
2018-01-03T22:07:00.684+0000 no indexes to restore
2018-01-03T22:07:00.684+0000 finished restoring kernel-ci.boot_regressions (144 documents)
2018-01-03T22:07:00.684+0000 reading metadata for kernel-ci.api-token from /tmp/kernelci_db_dump/kernel-ci/api-token.metadata.json
2018-01-03T22:07:00.684+0000 restoring kernel-ci.api-token from /tmp/kernelci_db_dump/kernel-ci/api-token.bson
2018-01-03T22:07:00.697+0000 no indexes to restore
2018-01-03T22:07:00.697+0000 finished restoring kernel-ci.boot_regressions_by_boot_id (297 documents)
2018-01-03T22:07:00.697+0000 reading metadata for kernel-ci.lab from /tmp/kernelci_db_dump/kernel-ci/lab.metadata.json
2018-01-03T22:07:00.698+0000 restoring kernel-ci.lab from /tmp/kernelci_db_dump/kernel-ci/lab.bson
2018-01-03T22:07:00.704+0000 restoring indexes for collection kernel-ci.api-token from metadata
2018-01-03T22:07:00.707+0000 finished restoring kernel-ci.api-token (6 documents)
2018-01-03T22:07:00.707+0000 reading metadata for kernel-ci.daily_stats from /tmp/kernelci_db_dump/kernel-ci/daily_stats.metadata.json
2018-01-03T22:07:00.707+0000 restoring kernel-ci.daily_stats from /tmp/kernelci_db_dump/kernel-ci/daily_stats.bson
2018-01-03T22:07:00.719+0000 restoring indexes for collection kernel-ci.daily_stats from metadata
2018-01-03T22:07:00.724+0000 finished restoring kernel-ci.daily_stats (0 documents)
2018-01-03T22:07:00.724+0000 reading metadata for kernel-ci.job from /tmp/kernelci_db_dump/kernel-ci/job.metadata.json
2018-01-03T22:07:00.724+0000 restoring kernel-ci.job from /tmp/kernelci_db_dump/kernel-ci/job.bson
2018-01-03T22:07:00.730+0000 restoring indexes for collection kernel-ci.lab from metadata
2018-01-03T22:07:00.732+0000 restoring indexes for collection kernel-ci.job from metadata
2018-01-03T22:07:00.732+0000 finished restoring kernel-ci.lab (2 documents)
2018-01-03T22:07:00.732+0000 reading metadata for kernel-ci.report from /tmp/kernelci_db_dump/kernel-ci/report.metadata.json
2018-01-03T22:07:00.732+0000 restoring kernel-ci.report from /tmp/kernelci_db_dump/kernel-ci/report.bson
2018-01-03T22:07:00.734+0000 finished restoring kernel-ci.job (0 documents)
2018-01-03T22:07:00.734+0000 reading metadata for kernel-ci.build from /tmp/kernelci_db_dump/kernel-ci/build.metadata.json
2018-01-03T22:07:00.743+0000 restoring kernel-ci.build from /tmp/kernelci_db_dump/kernel-ci/build.bson
2018-01-03T22:07:00.746+0000 restoring indexes for collection kernel-ci.build from metadata
2018-01-03T22:07:00.746+0000 restoring indexes for collection kernel-ci.report from metadata
2018-01-03T22:07:00.762+0000 restoring indexes for collection kernel-ci.boot from metadata
2018-01-03T22:07:00.762+0000 finished restoring kernel-ci.report (0 documents)
2018-01-03T22:07:00.762+0000 reading metadata for kernel-ci.bisect from /tmp/kernelci_db_dump/kernel-ci/bisect.metadata.json
2018-01-03T22:07:00.762+0000 restoring kernel-ci.bisect from /tmp/kernelci_db_dump/kernel-ci/bisect.bson
2018-01-03T22:07:00.764+0000 finished restoring kernel-ci.build (0 documents)
2018-01-03T22:07:00.764+0000 reading metadata for kernel-ci.error_logs from /tmp/kernelci_db_dump/kernel-ci/error_logs.metadata.json
2018-01-03T22:07:00.764+0000 restoring kernel-ci.error_logs from /tmp/kernelci_db_dump/kernel-ci/error_logs.bson
2018-01-03T22:07:00.768+0000 restoring indexes for collection kernel-ci.bisect from metadata
2018-01-03T22:07:00.768+0000 restoring indexes for collection kernel-ci.error_logs from metadata
2018-01-03T22:07:00.769+0000 finished restoring kernel-ci.boot (959 documents)
2018-01-03T22:07:00.772+0000 finished restoring kernel-ci.error_logs (0 documents)
2018-01-03T22:07:00.776+0000 finished restoring kernel-ci.bisect (0 documents)
2018-01-03T22:07:00.992+0000 no indexes to restore
2018-01-03T22:07:00.992+0000 finished restoring kernel-ci.test_case (40517 documents)
2018-01-03T22:07:00.992+0000 done
+ exit 0

FYI: I am now working on how to backup/restore logs from a kCI instance to another.

Apologies, it seemed like there was a bug remaining in the seed.sh script causing issues while restoring the database.I have fixed it and improved the seed.sh script feedback to give relevant information on what's happening / what failed: https://github.com/lucj/kernelci-docker/commit/e6910f7743f22d119c24c877e19ee9d9f25327c0

You can also find attached a log on how I restarted a kernelci-docker instance on an already existing machine. I have used JSM tarball file so you should be able to use exactly the same commands. If you are doing a fresh setup just skip the first command: ./stop.sh
 
Concerning the caching, I would recommend that you restore the database first before going to the frontend URL in a browser. Then you can look if the documents are present. In that order all the tests documents should appear right away once the database is restored.
 
 

Output from Ryan's run: