Compare commits

...

76 Commits

Author SHA1 Message Date
spike
1415b9208f update gitian version and patch, fix make vms 2017-04-10 17:25:22 +02:00
gdm85
9c1b649eb4 Minimal dmd+dub image 2016-11-09 23:42:47 +01:00
gdm85
7a92ae8858 Most recent version (with module parameters)
Fixed an issue with inspecting a base image
2015-11-24 13:03:00 +01:00
gdm85
49d951e855 Latest fixes for Mac OS 12,1 bluetooth patch 2015-11-24 10:45:31 +01:00
gdm85
6cbfaa595c Script to build .deb kernel packages to workaround Bluetooth issue
(patch as described in http://www.spinics.net/lists/linux-bluetooth/msg64111.html)
2015-10-28 16:06:43 +01:00
gdm85
2381e4d8db Ignore packages and patches subdirectories 2015-10-28 16:06:43 +01:00
gdm85
e7eacc9d2e Adding templating to support multiple Ubuntu versions
Renaming images to be more generic and not tied to Trusty

Have both pkgbuilder and kernelbuilder support multiple versions of Ubuntu
2015-10-28 16:06:43 +01:00
gdm85
db73573b58 Allow user 'nobody' to write in /tmp
Make /tmp home directory for user 'nobody'
2015-09-26 10:41:50 +02:00
gdm85
f5ddf1062e Imported rkt-images into tenku 2015-09-26 10:04:20 +02:00
gdm85
93b96bc594 Imported lxd-images into tenku 2015-09-26 10:04:14 +02:00
gdm85
7c1075116f Split basic package builder from kernel builder 2015-03-31 14:08:53 +02:00
gdm85
ed48df5a4e Don't try to delete non-existing block device /dev/kvm 2015-02-12 17:58:19 +01:00
gdm85
b81fb3ac93 Moved docker-iptables and docker-ports to docker-cli-tools 2015-01-20 13:07:25 +01:00
gdm85
2a797d8323 Add environment option to not dispose containers (NOPURGE)
Improve in-comments documentation for various options
2015-01-16 20:26:48 +01:00
gdm85
0673b29d1b Disable possibility of using /dev/kvm in containers
Handle case of "null" JSON value
Fix default behavior with no CLI arguments
2015-01-16 14:53:31 +01:00
gdm85
3f2871a4e5 Added workaround for bug @stedolan/jq/671 2015-01-16 11:07:41 +01:00
gdm85
3aaaa2e188 Better output directory format (human readable)
Use commits SHA hashes as unique references
2015-01-16 11:05:12 +01:00
gdm85
0f0c14a80a Fix parallel build for multiple OSes 2015-01-14 18:15:10 +01:00
gdm85
41f182c9c7 Provide also asserts for detached signatures 2015-01-14 15:20:02 +01:00
gdm85
900f80ec76 Fix debug typo 2015-01-14 10:49:02 +01:00
gdm85
92b5ae82c8 Fix permission issues
Use local and container path variables
2015-01-14 10:49:02 +01:00
gdm85
b2f83fcc9c Do not support AUFS
Some permissions fixes
2015-01-14 10:49:02 +01:00
gdm85
7bd75cc4b4 Completed automation for unattended gitian bitcoin builds 2015-01-14 10:49:02 +01:00
gdm85
8f79fccb14 First draft for automated latest version builds 2015-01-14 10:49:02 +01:00
gdm85
768db46d28 docker-iplock -> replaced by docker-fw 2015-01-14 10:49:02 +01:00
gdm85
6fd0ef5799 docker-hosts -> replaced by docker-cli-tools/docker-hosts 2015-01-14 10:49:02 +01:00
gdm85
c9ff828e51 Added missing --commit 2015-01-14 10:49:02 +01:00
gdm85
a660cb4367 Fixes to integrate with new bitcoin core dependency management & cache 2015-01-14 10:49:02 +01:00
gdm85
5407f858d4 Update README.md 2015-01-14 10:49:02 +01:00
gdm85
6bfe823cc7 Added inputs for 0.9.3rc2
Locked gitian-builder to working commit
Removed nl_NL.UTF-8 from locale.gen
Do not overwrite already existing downloaded input sources
Use COPY in place of ADD in Dockerfile's
2015-01-14 10:49:02 +01:00
gdm85
59d49e1360 Added script to fix auto-published Docker container ports on host 2015-01-14 10:49:02 +01:00
gdm85
85b459ffeb Script to show summary of allocated ports of a container 2015-01-14 10:49:02 +01:00
gdm85
550af3bd51 Typo fixes/wording improvements 2015-01-14 10:49:02 +01:00
gdm85
76b7213a32 A few syntax fixes 2015-01-14 10:49:02 +01:00
Michael Ford
6ace40f2e0 Correct link to gitian bitcoin host container 2015-01-14 10:49:02 +01:00
gdm85
1583dac1ed * added descriptors/inputs/hashes for 0.9.2 (same as 0.9.2rc2)
* fixed issue with existing input removal
* remove part of patch that was merged upstream
* more consistent check for gitian host VM creation
2015-01-14 10:49:02 +01:00
gdm85
dda351daac * always remove destination of input downloads
* fixed move command after gbuilds
2015-01-14 10:49:02 +01:00
gdm85
2bb9aab208 * fixed gitian patch (line was truncated)
* sleep after docker commit
2015-01-14 10:49:02 +01:00
gdm85
aec8437a1d * moved custom gitian patch to gitian-host
* avoid chown long time operation by chown'ing specific files
* use correct list filenames
* removed libpng (not used for Linux builds)
* use docker kill instead of stop
2015-01-14 10:49:02 +01:00
gdm85
3ff080d312 * support different descriptors across versions 2015-01-14 10:49:02 +01:00
gdm85
23450a3027 * parallelize wget downloads of inputs
* removed extra environment cleanup from bash_profile and added proper gitian-builder patch
* use gitian-host-vms for gitian-bitcoin-host
* fixed issue at stopping container after generation of base VMs
2015-01-14 10:49:02 +01:00
gdm85
eee1af1e61 * added sources for 0.9.2rc2 2015-01-14 10:49:02 +01:00
gdm85
9e50b80864 * ask for answer on same line 2015-01-14 10:49:02 +01:00
gdm85
de4970e14c * script for creation of trusty-kbuilder image 2015-01-14 10:49:02 +01:00
gdm85
0e216f2548 * added Ubuntu Archive reference keyring
* added script to debootstrap Ubuntu Trusty
* added Dockerfile for Ubuntu Trusty kernel builder
2015-01-14 10:49:02 +01:00
gdm85
a05c460c2d * removed reference to original article (dead) 2015-01-14 10:49:02 +01:00
gdm85
a3d8d7c755 * added Debian Archive keyring
* added questions to user about keyring differences/trust
* added documentation notes about the Debian Archive keyring
2015-01-14 10:49:02 +01:00
gdm85
0593a6b372 * conceal also docker host IP in docker-iptables
* added docker-hosts for simplified hosts listing
2015-01-14 10:49:02 +01:00
gdm85
05ee260412 * added generation of gitian-host-vms image
* simplified necessary steps
* fixed issue with parallel package missing
2015-01-14 10:49:02 +01:00
gdm85
66e188ab57 * add message about location of output files 2015-01-14 10:49:02 +01:00
gdm85
19bc999e55 * verify md5 hashes before using files for gbuild 2015-01-14 10:49:02 +01:00
gdm85
6f983eecb1 * added missing file 2015-01-14 10:49:02 +01:00
gdm85
35611fb24e * proper fix for SSH environment pollution (see https://bugzilla.mindrot.org/show_bug.cgi?id=1285) 2015-01-14 10:49:02 +01:00
gdm85
823f62f641 * ignore failure at deleting (unknown reason) 2015-01-14 10:49:02 +01:00
gdm85
afc8353fe0 * added workaround for Docker glitch with file permissions
* a few improvements to documentation, describing steps needed to build bitcoin
2015-01-14 10:49:02 +01:00
gdm85
bb1d304753 * unused file 2015-01-14 10:49:02 +01:00
gdm85
bf46c0c95a * added support for newer versions of bitcoin 2015-01-14 10:49:02 +01:00
gdm85
2cd7b6b6c0 * added preamble 2015-01-14 10:49:02 +01:00
gdm85
a31c97fb5a * fix typo & enlarge 2015-01-14 10:49:02 +01:00
gdm85
c3ed3cf2d7 * added (horrible) diagram 2015-01-14 10:49:02 +01:00
gdm85
7f16f8315f * improve readability 2015-01-14 10:48:59 +01:00
gdm85
8f3167f9e8 * improvements/fix typos 2015-01-14 10:48:06 +01:00
gdm85
5a444d84a8 * added documentation & scripts for full automation of gitian bitcoin builds 2015-01-14 10:48:06 +01:00
gdm85
22c9a8a461 * put base VM creation script in gitian-host itself 2015-01-14 10:48:06 +01:00
gdm85
c836623f7a * specific documentation for building bitcoin with gitian-host 2015-01-14 10:48:06 +01:00
gdm85
b616daa07b * bitcoin gitian-host scripts (0.9.1) 2015-01-14 10:48:06 +01:00
gdm85
2f6956eb88 * added missing patch
* added changes for previous commit
2015-01-14 10:48:06 +01:00
gdm85
1d4b24aec1 * added documentation about building bitcoin 0.9.1
* a few fixes to scripts to overcome docker glitches
2015-01-14 10:48:06 +01:00
gdm85
3491c6646e * completed documentation
* added prefix to image tags
2015-01-14 10:48:06 +01:00
gdm85
606e441cd3 * final touches 2015-01-14 10:48:06 +01:00
gdm85
51cd62186d * first draft of scripts for gitian host creation 2015-01-14 10:48:06 +01:00
gdm85
29bacde903 * adding scripts for creation/spawning of gitian hosts 2015-01-14 10:48:06 +01:00
gdm85
640dd591fe * script to build a base Debian Wheezy docker image 2015-01-14 10:48:06 +01:00
gdm85
a78ecfc818 * added README.md
* reorganizing
2015-01-14 10:48:06 +01:00
gdm85
74ee17c3ae * a few improvements 2015-01-14 10:48:05 +01:00
gdm85
4f8b3ff044 * skip containers with no network 2015-01-14 10:48:01 +01:00
58 changed files with 1843 additions and 24 deletions

4
.gitignore vendored Normal file
View File

@ -0,0 +1,4 @@
rkt/nginx/nginx.aci
lxd/busybox-nonroot/busybox-nonroot.tar.xz
docker/ubuntu-kernelbuilder/packages

7
README.md Normal file
View File

@ -0,0 +1,7 @@
gdm85's bay
===========
This repository contains various scripts and documentation I have gathered.
<a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-sa/2.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/">Creative Commons Attribution-ShareAlike 2.0 Generic License</a>.

View File

@ -1,24 +0,0 @@
#!/bin/bash
## docker-iptables
##
## @author gdm85
## script to show iptables rules with docker names
## can be used also to detect problems with dead containers and stale iptable rules
## supports standard iptables-save syntax
#
function replace_iptables() {
local CID
local SEDCMD=""
for CID in $(docker ps -q -a); do
local NAME=$(docker inspect --format '{{ .Name }}' $CID | awk '{ print substr($0, 2, length($0)-1) }')
local IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $CID)
SEDCMD="$SEDCMD -e s!$IP/32!${NAME}!g"
done
sed $SEDCMD
}
iptables-save $@ | replace_iptables

18
docker/dmd/Dockerfile Normal file
View File

@ -0,0 +1,18 @@
## dmd
##
## VERSION 2.072
##
## a minimal image installing the official .deb for dmd
## the official image would instead compile and install too many dependencies
##
#
## this image must be debootstrapped with build-wheezy.sh
FROM gdm85/wheezy
## install deps
RUN export DEBIAN_FRONTEND=noninteractive && apt-get update -y && apt-get install libc6-dev gcc libcurl3 -y
## install dmd+dub
RUN export DEBIAN_FRONTEND=noninteractive && cd /tmp && wget --quiet http://downloads.dlang.org/releases/2.x/2.072.0/dmd_2.072.0-0_amd64.deb && \
echo 'cd61bc9d5e511db5d35b3e1410c21f0d38941e038e06c6db2db18e73177efbae dmd_2.072.0-0_amd64.deb' | sha256sum -c && dpkg -i dmd_2.072.0-0_amd64.deb && rm dmd_2.072.0-0_amd64.deb

View File

@ -0,0 +1,28 @@
## gitian-bitcoin-host
##
## VERSION 0.1.0
##
FROM gdm85/gitian-host-vms
MAINTAINER Giuseppe Mazzotta "gdm85@users.noreply.github.com"
RUN aptitude install make -y
COPY input-sources/ /home/debian/input-sources/
COPY bin/build-bitcoin.sh /home/debian/
COPY bin/sign.sh /home/debian/
## patch to not attempt removal of build directory (will be a volume)
COPY gitian-gbuild.patch /home/debian/
RUN cd gitian-builder && patch -p1 < ../gitian-gbuild.patch && rm ../gitian-gbuild.patch
## patch to write asserts only (best for detached signatures)
COPY gitian-gasserts.patch /home/debian/
RUN cd gitian-builder && cp bin/gsign bin/gasserts && patch -p1 < ../gitian-gasserts.patch && rm ../gitian-gasserts.patch
WORKDIR /home/debian
RUN mkdir gitian-builder/inputs
RUN chown -R debian.debian input-sources build-bitcoin.sh sign.sh gitian-builder/inputs

View File

@ -0,0 +1,64 @@
Building bitcoin with a gitian-builder Docker container
=======================================================
This image allows automated Gitian builds of bitcoin core using a Docker container.
Prerequisites
-------------
Before proceeding make sure you have created these necessary images:
* gdm85/wheezy
* gdm85/gitian-host
* gdm85/gitian-host-vms
Instructions to build them are available [here](../gitian-host/README.md).
Image creation
---------------
Afterwards you can create the *gdm85/gitian-bitcoin-host* image by running [scripts/create-gitian-bitcoin-host.sh](../scripts/create-gitian-bitcoin-host.sh).
Bitcoin input sources
---------------------
Since version 0.10.0 it's no more needed to use [input-sources/](input-sources/).
Do not forget to read also the [Preamble here](../gitian-host/README.md#preamble) to correctly use Gitian builder and provided scripts.
Spawning a container
--------------------
A new container for Gitian bitcoin builds can be spawned with:
- [scripts/spawn-gitian-bitcoin-host.sh](scripts/spawn-gitian-bitcoin-host.sh)
This script will spawn a running Docker container and provide details about how to connect via SSH to the container, example:
```
$ scripts/spawn-gitian-bitcoin-host.sh
You can now SSH into container 3bc0d0611374ca4d4730fd5fb1067808b1bcfd072ec7cf029393a7fd99ec856e:
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@172.17.0.3
$
```
You can use this specific SSH command line to get a shell in the container and proceed to next steps.
Preparing the gitian environment
--------------------------------
To initiate a gitian build of bitcoin you will run:
- [./build-bitcoin.sh](bin/build-bitcoin.sh) 0.10.0rc1
Notice the parameter 0.10.0, that is the version we are going to build and must be available in [input-sources](input-sources/).
[build-bitcoin.sh](bin/build-bitcoin.sh) is a script that will download & build all the dependencies and then bitcoin itself, for both i386 and amd64 Linux architectures.
Signing
-------
In order to sign the build you can either import your private key in container's debian user gpg, or perform the signing externally.
In this example we will cover the former case; run [~/sign.sh](bin/sign.sh) script and check that your generated assert file (in a subdirectory of ~/gitian.sigs) matches with those of [other developers](https://github.com/bitcoin/gitian.sigs).
**NOTE:** Only the out_manifest signatures do matter, not all the dependencies.
Submitting your signature
-------------------------
If signatures do match, you can fork the [gitian sigs repo](https://github.com/bitcoin/gitian.sigs), add & commit your signatures and submit a pull request for inclusion.

View File

@ -0,0 +1,82 @@
#!/bin/bash
## automatic Gitian build of bitcoin
## @author gdm85
## @version 0.3.1
## see also https://github.com/gdm85/tenku/blob/master/docker/gitian-bitcoin-host/
##
#
if [ $# -lt 2 ]; then
echo "Usage: build-bitcoin.sh commit linux [win] [osx] [...]" 1>&2
exit 1
fi
COMMIT="$1"
shift
## remaining parameters are OS targets to be build (e.g. win,osx,linux)
CLONE="$HOME/bitcoin"
function verlte() {
[ "$1" = "`echo -e "$1\n$2" | sort -V | head -n1`" ]
}
NPROC=$(nproc) && \
cd gitian-builder && \
mkdir -p inputs && \
cd .. || exit $?
if [ ! -d bitcoin ]; then
git clone https://github.com/bitcoin/bitcoin.git && \
cd bitcoin && \
git checkout $COMMIT && \
cd .. || exit $?
fi
## old logic using descriptors (only linux supported)
if echo "$COMMIT" | grep ^v >/dev/null && ! verlte v0.10.0rc1 $COMMIT; then
## make sure only Linux is being built
if [[ ! $# -eq 1 && "$1" != "linux" ]]; then
echo "For versions before 0.10.0rc1, only Linux building is supported" 1>&2
exit 1
fi
VERSION=$(echo "$COMMIT" | awk '{ print substr($0, 2) }')
cd gitian-builder/inputs || exit $?
## get each dependency
## they are validated afterwards by gbuild
while read -r URL FNAME; do
if [ -z "$URL" ]; then
continue
fi
if [ ! -f $FNAME ]; then
echo "echo 'Downloading $FNAME'"
echo "wget -q --no-check-certificate '$URL' -O '$FNAME' || echo 'Failed to download $FNAME from $URL'"
fi
done < ../../input-sources/${VERSION}-inputs.txt | parallel -j10 || exit $?
## verify that all sources are correct before continuing
md5sum -c < ../../input-sources/${VERSION}-inputs.md5 && \
DESCRIPTORS="$(<../input-sources/${VERSION}-descriptors.txt)" && \
cd .. || exit $?
for DESC in $DESCRIPTORS; do
./bin/gbuild -j$NPROC ../bitcoin/contrib/gitian-descriptors/${DESC}.yml && \
mv -v $(find build/out -type f -name '*gz' -o -name '*.zip') inputs/ || exit $?
done
else
cd bitcoin/depends || exit $?
for DESC in $@; do
make download-${DESC} SOURCES_PATH="$HOME/gitian-builder/cache/common" || exit $?
done
cd ../.. || exit $?
fi
## proceed to build of each of the specified gitian descriptors
cd gitian-builder || exit $?
for DESC in $@; do
./bin/gbuild -j$NPROC --commit bitcoin=$COMMIT -u bitcoin=$CLONE "$CLONE/contrib/gitian-descriptors/gitian-${DESC}.yml" || exit $?
done
echo "Successfully built gitian-${DESC} at $COMMIT"

View File

@ -0,0 +1,13 @@
#!/bin/bash
if [ ! $# -eq 3 ]; then
echo "Usage: sign.sh version signer-id gitian-descriptor.yml" 1>&2
exit 1
fi
VERSION="$1"
SIGNER="$2"
DESC="$3"
cd gitian-builder && \
./bin/gsign --signer $SIGNER --release ${VERSION} --destination ../gitian.sigs/ "../bitcoin/contrib/gitian-descriptors/$DESC"

View File

@ -0,0 +1,8 @@
--- a/bin/gasserts 2015-01-13 15:39:29.000000000 +0000
+++ b/bin/gasserts 2015-01-14 13:52:49.287255738 +0000
@@ -85,4 +85,3 @@
File.open(assert_path, "w") do |io|
io.write result.to_yaml
end
-system!("gpg --detach-sign -u \"#{signer}\" \"#{assert_path}\"")

View File

@ -0,0 +1,13 @@
--- a/bin/gbuild 2015-01-14 10:25:27.275143892 +0100
+++ b/bin/gbuild 2015-01-14 10:25:56.171144093 +0100
@@ -182,8 +182,8 @@
cache_dir = 'cache'
enable_cache = build_desc["enable_cache"]
-FileUtils.rm_rf(build_dir)
-FileUtils.mkdir(build_dir)
+## build dir is cleaned up outside container running gitian-builder
+FileUtils.mkdir_p(build_dir)
FileUtils.mkdir_p(result_dir)
package_name = build_desc["name"] or raise "must supply name"

View File

@ -0,0 +1,2 @@
boost-linux
deps-linux

View File

@ -0,0 +1,8 @@
d6eef4b4cacb2183f2bf265a5a03a354 boost_1_55_0.tar.bz2
a14a5486d6b4891d2434039a0ed4c5b7 db-4.8.30.NC.tar.gz
065bf20a20ebe605c675b7a5aaef340a miniupnpc-1.8.tar.gz
de62b43dfcd858e66a74bee1c834e959 openssl-1.0.1g.tar.gz
a72001a9067a4c2c4e0e836d0f92ece4 protobuf-2.5.0.tar.bz2
89a90a3b8cbca60ac412b2e0e0c776e7 qrencode-3.4.3.tar.bz2
228b6384dfd7272de00fd8b2c144fecd qt-everywhere-opensource-src-5.2.0.tar.gz
44d667c142d7cda120332623eab69f40 zlib-1.2.8.tar.gz

View File

@ -0,0 +1,8 @@
http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.8.tar.gz miniupnpc-1.8.tar.gz
https://www.openssl.org/source/openssl-1.0.1g.tar.gz openssl-1.0.1g.tar.gz
http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz db-4.8.30.NC.tar.gz
http://zlib.net/zlib-1.2.8.tar.gz zlib-1.2.8.tar.gz
https://fukuchi.org/works/qrencode/qrencode-3.4.3.tar.bz2 qrencode-3.4.3.tar.bz2
https://downloads.sourceforge.net/project/boost/boost/1.55.0/boost_1_55_0.tar.bz2 boost_1_55_0.tar.bz2
https://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.gz qt-everywhere-opensource-src-5.2.0.tar.gz
https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2 protobuf-2.5.0.tar.bz2

View File

@ -0,0 +1,3 @@
boost-linux
deps-linux
qt-linux

View File

@ -0,0 +1,11 @@
d6eef4b4cacb2183f2bf265a5a03a354 boost_1_55_0.tar.bz2
efe08e2f3ca478486037b053acd512e9 cdrkit-1.1.11.tar.gz
cea2d01b3206e92a8df7b079935c070b clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz
a14a5486d6b4891d2434039a0ed4c5b7 db-4.8.30.NC.tar.gz
5ef3ba321e6df72d6519b728b292073e miniupnpc-1.9.tar.gz
8d6d684a9430d5cc98a62a5d8fbda8cf openssl-1.0.1h.tar.gz
a72001a9067a4c2c4e0e836d0f92ece4 protobuf-2.5.0.tar.bz2
89a90a3b8cbca60ac412b2e0e0c776e7 qrencode-3.4.3.tar.bz2
8ac880cc07a130c39607b65efd5e1421 qt-everywhere-opensource-src-4.6.4.tar.gz
228b6384dfd7272de00fd8b2c144fecd qt-everywhere-opensource-src-5.2.0.tar.gz
44d667c142d7cda120332623eab69f40 zlib-1.2.8.tar.gz

View File

@ -0,0 +1,11 @@
http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.9.tar.gz miniupnpc-1.9.tar.gz
https://www.openssl.org/source/openssl-1.0.1h.tar.gz openssl-1.0.1h.tar.gz
http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz db-4.8.30.NC.tar.gz
http://zlib.net/zlib-1.2.8.tar.gz zlib-1.2.8.tar.gz
https://fukuchi.org/works/qrencode/qrencode-3.4.3.tar.bz2 qrencode-3.4.3.tar.bz2
https://downloads.sourceforge.net/project/boost/boost/1.55.0/boost_1_55_0.tar.bz2 boost_1_55_0.tar.bz2
https://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.gz qt-everywhere-opensource-src-5.2.0.tar.gz
https://download.qt-project.org/archive/qt/4.6/qt-everywhere-opensource-src-4.6.4.tar.gz qt-everywhere-opensource-src-4.6.4.tar.gz
https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2 protobuf-2.5.0.tar.bz2
http://cdrkit.org/releases/cdrkit-1.1.11.tar.gz cdrkit-1.1.11.tar.gz
http://llvm.org/releases/3.2/clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz

View File

@ -0,0 +1,3 @@
boost-linux
deps-linux
qt-linux

View File

@ -0,0 +1,11 @@
d6eef4b4cacb2183f2bf265a5a03a354 boost_1_55_0.tar.bz2
efe08e2f3ca478486037b053acd512e9 cdrkit-1.1.11.tar.gz
cea2d01b3206e92a8df7b079935c070b clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz
a14a5486d6b4891d2434039a0ed4c5b7 db-4.8.30.NC.tar.gz
5ef3ba321e6df72d6519b728b292073e miniupnpc-1.9.tar.gz
8d6d684a9430d5cc98a62a5d8fbda8cf openssl-1.0.1h.tar.gz
a72001a9067a4c2c4e0e836d0f92ece4 protobuf-2.5.0.tar.bz2
89a90a3b8cbca60ac412b2e0e0c776e7 qrencode-3.4.3.tar.bz2
8ac880cc07a130c39607b65efd5e1421 qt-everywhere-opensource-src-4.6.4.tar.gz
228b6384dfd7272de00fd8b2c144fecd qt-everywhere-opensource-src-5.2.0.tar.gz
44d667c142d7cda120332623eab69f40 zlib-1.2.8.tar.gz

View File

@ -0,0 +1,11 @@
http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.9.tar.gz miniupnpc-1.9.tar.gz
https://www.openssl.org/source/openssl-1.0.1h.tar.gz openssl-1.0.1h.tar.gz
http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz db-4.8.30.NC.tar.gz
http://zlib.net/zlib-1.2.8.tar.gz zlib-1.2.8.tar.gz
https://fukuchi.org/works/qrencode/qrencode-3.4.3.tar.bz2 qrencode-3.4.3.tar.bz2
https://downloads.sourceforge.net/project/boost/boost/1.55.0/boost_1_55_0.tar.bz2 boost_1_55_0.tar.bz2
https://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.gz qt-everywhere-opensource-src-5.2.0.tar.gz
https://download.qt-project.org/archive/qt/4.6/qt-everywhere-opensource-src-4.6.4.tar.gz qt-everywhere-opensource-src-4.6.4.tar.gz
https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2 protobuf-2.5.0.tar.bz2
http://cdrkit.org/releases/cdrkit-1.1.11.tar.gz cdrkit-1.1.11.tar.gz
http://llvm.org/releases/3.2/clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz

View File

@ -0,0 +1,3 @@
boost-linux
deps-linux
qt-linux

View File

@ -0,0 +1,10 @@
d6eef4b4cacb2183f2bf265a5a03a354 boost_1_55_0.tar.bz2
cea2d01b3206e92a8df7b079935c070b clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz
a14a5486d6b4891d2434039a0ed4c5b7 db-4.8.30.NC.tar.gz
56b4ac6a7a3e7d64f8cfeeac87b212bc miniupnpc-1.9.20140701.tar.gz
c8dc151a671b9b92ff3e4c118b174972 openssl-1.0.1i.tar.gz
a72001a9067a4c2c4e0e836d0f92ece4 protobuf-2.5.0.tar.bz2
89a90a3b8cbca60ac412b2e0e0c776e7 qrencode-3.4.3.tar.bz2
8ac880cc07a130c39607b65efd5e1421 qt-everywhere-opensource-src-4.6.4.tar.gz
228b6384dfd7272de00fd8b2c144fecd qt-everywhere-opensource-src-5.2.0.tar.gz
44d667c142d7cda120332623eab69f40 zlib-1.2.8.tar.gz

View File

@ -0,0 +1,10 @@
http://miniupnp.free.fr/files/download.php?file=miniupnpc-1.9.20140701.tar.gz miniupnpc-1.9.20140701.tar.gz
https://www.openssl.org/source/openssl-1.0.1i.tar.gz openssl-1.0.1i.tar.gz
http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz db-4.8.30.NC.tar.gz
http://zlib.net/zlib-1.2.8.tar.gz zlib-1.2.8.tar.gz
https://fukuchi.org/works/qrencode/qrencode-3.4.3.tar.bz2 qrencode-3.4.3.tar.bz2
https://downloads.sourceforge.net/project/boost/boost/1.55.0/boost_1_55_0.tar.bz2 boost_1_55_0.tar.bz2
https://download.qt-project.org/official_releases/qt/5.2/5.2.0/single/qt-everywhere-opensource-src-5.2.0.tar.gz qt-everywhere-opensource-src-5.2.0.tar.gz
https://download.qt-project.org/archive/qt/4.6/qt-everywhere-opensource-src-4.6.4.tar.gz qt-everywhere-opensource-src-4.6.4.tar.gz
https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.bz2 protobuf-2.5.0.tar.bz2
http://llvm.org/releases/3.2/clang+llvm-3.2-x86-linux-ubuntu-12.04.tar.gz clang-llvm-3.2-x86-linux-ubuntu-12.04.tar.gz

View File

@ -0,0 +1,74 @@
## gitian-host
##
## VERSION 0.1.0
##
## gitian host privileged container
## see also https://github.com/gdm85/tenku/tree/master/docker/gitian-host/README.md
##
#
## this image must be debootstrapped with build-wheezy.sh
FROM gdm85/wheezy
MAINTAINER Giuseppe Mazzotta "gdm85@users.noreply.github.com"
## some core packages
RUN apt-get update && apt-get install -y aptitude
RUN aptitude update && aptitude safe-upgrade -y
RUN aptitude install htop tmux less locales openssh-server -y
## these 2 files can be customized
COPY etc/timezone /etc/
COPY etc/locale.gen /etc/
RUN dpkg-reconfigure locales && dpkg-reconfigure tzdata
## continue installation of gitian-builder packages
RUN echo "deb http://http.debian.net/debian wheezy main contrib non-free" > /etc/apt/sources.list.d/non-free.list
RUN apt-get install -y iptables ca-certificates sudo bridge-utils git python-cheetah qemu-utils parted kpartx ruby apt-cacher-ng lxc parallel patch
## disable password logins, reverse DNS lookups & privilege separation (due to chroot issues on latest docker)
RUN sed -i -e 's/^#PasswordAuthentication yes/PasswordAuthentication no/' -e 's/^UsePrivilegeSeparation.*$/UsePrivilegeSeparation no/' /etc/ssh/sshd_config && echo 'UseDNS no' >> /etc/ssh/sshd_config
EXPOSE 22
# this version of Debian needs to run as root, so make sure
# that the build script can exectute it without providing a password
RUN echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-start" > /etc/sudoers.d/gitian-lxc
RUN echo 'debian ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/debian-sudo
RUN useradd -m -s /bin/bash debian && adduser debian sudo && mkdir /home/debian/.ssh
## provide authorized keys to allow connections inside host
## NOTE: this is placed in debian user directory by setup.sh script
## reason is that there is a Docker glitch with file permissions when adding this
COPY authorized_keys /root/
WORKDIR /home/debian
## download in clear HTTP & check md5 right afterwards
RUN wget http://archive.ubuntu.com/ubuntu/pool/universe/v/vm-builder/vm-builder_0.12.4+bzr489.orig.tar.gz && echo "ec12e0070a007989561bfee5862c89a32c301992dd2771c4d5078ef1b3014f03 vm-builder_0.12.4+bzr489.orig.tar.gz" | sha256sum -c
RUN tar -zxvf vm-builder_0.12.4+bzr489.orig.tar.gz && cd vm-builder-0.12.4+bzr489 && python setup.py install
## these must always be loaded when you login as 'debian' user
COPY bash_profile /home/debian/.bash_profile
## script used to build base VMs
COPY bin/build-base-vms.sh /home/debian/
RUN git clone https://github.com/devrandom/gitian-builder.git && cd gitian-builder && git checkout ad3f9cc4c2c8c0899961a366f5b9fbd1483b0ee3
COPY gitian.patch /home/debian/
## patch to allow paralle creation of VMs
RUN cd gitian-builder && patch -p1 < ../gitian.patch && rm ../gitian.patch
RUN chown -R debian.debian .bash_profile . && chown -R apt-cacher-ng.apt-cacher-ng /var/cache/apt-cacher-ng/
## suggested: run this image with mounted volumes for pre-downloaded inputs and apt-cacher-ng cache
## -v inputs:/home/debian/gitian-builder/inputs -v apt-cacher-ng:/var/cache/apt-cacher-ng
##NOTE: if you do use them, then not forget to set proper user rights for these mounted volumes
## all necessary setup instructions (SSH, LXC)
COPY bin/setup.sh /usr/local/bin/setup.sh
## on run failure, check logs to see if setup failed. on success connect via SSH
CMD /usr/local/bin/setup.sh && /etc/init.d/apt-cacher-ng start && /usr/sbin/sshd -D -e

View File

@ -0,0 +1,60 @@
Gitian host docker container
============================
The provided [Dockerfile](http://docs.docker.io/reference/builder/) allows to generate a [gitian-builder](https://gitian.org/) host image, that can subsequently be used for reproducible builds using LXC VMs.
How this works:
<img src="diagram.png">
Some of the discussions leading to the creation of this set of Dockerfiles/scripts are available on [this issue](https://github.com/devrandom/gitian-builder/issues/53).
Preamble
--------
It is **necessary** to read the provided scripts and understand what they do before starting to use them.
The reason is that aiming at the goal of creating a Gitian build (deterministic) also means that your build is untampered, thus trust shall be correctly attributed during the process.
For example, in this repository I provide the [Debian Archive keyring](../keyrings/debian-archive-keyring.gpg) that is used for the original debootstrap, however you **must** verify its authenticity and that it is exactly [as provided officially by Debian](https://packages.debian.org/wheezy/all/debian-archive-keyring/download)
in order to continue using a trusted chain of systems.
See also:
- https://gitian.org/
- https://en.wikipedia.org/wiki/Web_of_trust
- http://www.dwheeler.com/trusting-trust/
- https://www.debian.org/
- https://wiki.debian.org/SecureApt
- https://www.docker.io/
- http://www.ubuntu.com/
How to build the image
----------------------
Images have not been pushed to my [Docker Registry](https://index.docker.io/) account, this is on purpose because even if generated images have my repository prefix ('gdm85/') you are supposed to create them from scratch with the provided (auditable) scripts.
**NOTE:** you must have *debootstrap* on your real host to run this script successfully, and also make sure you have a keyring with APT keys, see also https://wiki.debian.org/SecureApt
First steps:
- run [scripts/build-wheezy.sh](../scripts/build-wheezy.sh) to get a Debian Wheezy image debootstrapped from Debian repositories.
- run [scripts/create-gitian-host.sh](../scripts/create-gitian-host.sh), this will simply build the Dockerfile that installs the few necessary dependencies inside the prepared image
- generate a second image with the i386 and amd64 VMs byu using [build-base-vms.sh](../scripts/build-base-vms.sh)
**NOTE:** when I say "run", what I really mean is "read the script, study it for your own learning purposes, then run it"
After steps above you will have prepared a full Gitian builder environment for deterministic builds.
The image that contains the VMs is called *gdm85/gitian-host-vms*; in future you can spawn containers with this image for new Gitian builder environments.
Example:
```
$ scripts/spawn-gitian-host.sh
You can now SSH into container 8a955ff5607b62d4c295745f27bbc38f2e8e011ea93053e641617d50ad2aa5a2:
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@172.17.0.2
$
```
This will create a privileged running container that you can access with the SSH command displayed.
Derived images
--------------
A [bitcoin gitian host container](../gitian-bitcoin-host/README.md) is available.
Credits
-------
Thanks to jpetazzo for [dind](https://github.com/jpetazzo/dind) and to #docker & bitcoin-dev IRC users for the help&assistance!

View File

@ -0,0 +1,3 @@
export USE_LXC=1
export GITIAN_HOST_IP=10.0.3.2
export LXC_GUEST_IP=10.0.3.5

View File

@ -0,0 +1,65 @@
#!/bin/bash
if [ $# -eq 0 ]; then
echo "Please specify: [i386|amd64]" 1>&2
exit 1
fi
if [ -z "$USE_LXC" ]; then
echo "Environment variables not correctly setup (source .bash_profile?)" 1>&2
exit 2
fi
export MIRROR_HOST=$GITIAN_HOST_IP
SUITE=precise
cd gitian-builder && \
mkdir -p var || exit $?
if [ ! -e var/id_dsa ]; then
ssh-keygen -t dsa -f var/id_dsa -N ""
fi
## build both VMs in parallel
for TYPE in "$@"; do
echo -e "MIRROR_HOST=$GITIAN_HOST_IP bin/make-base-vm --lxc --arch $TYPE --suite $SUITE"
done | parallel -j$# || exit $?
function retry_remove() {
local RETRIES="$1"
local LOOP="$2"
while ! sudo losetup -d "/dev/loop${LOOP}" 2>/dev/null; do
let RETRIES-=1
if [ $RETRIES -eq 0 ]; then
echo "Failed removing /dev/loop${LOOP}" 1>&2
return 1
fi
sleep 1
done
sudo unlink /dev/mapper/loop${LOOP}p1 2>/dev/null
}
## this function corresponds to part removed from gbuild via custom patch
function ext_partition() {
local loop
local OUT=$1
echo "Converting $OUT to raw format..." && \
qemu-img convert $OUT.qcow2 $OUT.raw && \
echo -n "Identifying partition..." && \
set -o pipefail && \
loop=`sudo kpartx -av $OUT.raw | sed -n '/loop.p1/{s/.*loop\(.\)p1.*/\1/;p}'` && \
echo ": $loop" && \
echo "Copying partition to $OUT..." && \
sudo cp --sparse=always /dev/mapper/loop${loop}p1 $OUT && \
sudo chown $USER $OUT || return $?
## these are silenced because if former fails, second doesn't and viceversa
echo "Removing partition loop mount..." && \
retry_remove 5 "$loop" && \
echo "Removing raw image..." && \
rm -f "$OUT.raw" && \
echo "$OUT correctly extracted"
}
for TYPE in "$@"; do
ext_partition base-${SUITE}-${TYPE} || exit $?
done

124
docker/gitian-host/bin/setup.sh Executable file
View File

@ -0,0 +1,124 @@
#!/bin/bash
## @author gdm85
## this script is an adapted version of jpetazzo's original:
## https://github.com/jpetazzo/dind/blob/master/wrapdocker
## (thus most of the comment are his voice)
#
## prevent (re)starting of sshd
## we want to use sshd as our container process
echo -e "#!/bin/sh\nexit 101" > /usr/sbin/policy-rc.d && \
chmod +x /usr/sbin/policy-rc.d
## regenerate host keys
/bin/rm -v /etc/ssh/ssh_host_* && \
dpkg-reconfigure -f noninteractive openssh-server || exit $?
## removed, in case you want to install other packages at container-time
rm /usr/sbin/policy-rc.d
# First, make sure that cgroups are mounted correctly.
CGROUP=/sys/fs/cgroup
[ -d $CGROUP ] ||
mkdir $CGROUP
mountpoint -q $CGROUP ||
mount -n -t tmpfs -o uid=0,gid=0,mode=0755 cgroup $CGROUP || {
echo "Could not make a tmpfs mount. Did you use -privileged?"
exit 1
}
if [ -d /sys/kernel/security ] && ! mountpoint -q /sys/kernel/security
then
mount -t securityfs none /sys/kernel/security || {
echo "Could not mount /sys/kernel/security."
echo "AppArmor detection and -privileged mode might break."
exit 2
}
fi
# Mount the cgroup hierarchies exactly as they are in the parent system.
for SUBSYS in $(cut -d: -f2 /proc/1/cgroup)
do
[ -d $CGROUP/$SUBSYS ] || mkdir $CGROUP/$SUBSYS
mountpoint -q $CGROUP/$SUBSYS ||
mount -n -t cgroup -o $SUBSYS cgroup $CGROUP/$SUBSYS
# The two following sections address a bug which manifests itself
# by a cryptic "lxc-start: no ns_cgroup option specified" when
# trying to start containers withina container.
# The bug seems to appear when the cgroup hierarchies are not
# mounted on the exact same directories in the host, and in the
# container.
# Named, control-less cgroups are mounted with "-o name=foo"
# (and appear as such under /proc/<pid>/cgroup) but are usually
# mounted on a directory named "foo" (without the "name=" prefix).
# Systemd and OpenRC (and possibly others) both create such a
# cgroup. To avoid the aforementioned bug, we symlink "foo" to
# "name=foo". This shouldn't have any adverse effect.
echo $SUBSYS | grep -q ^name= && {
NAME=$(echo $SUBSYS | sed s/^name=//)
ln -s $SUBSYS $CGROUP/$NAME
}
# Likewise, on at least one system, it has been reported that
# systemd would mount the CPU and CPU accounting controllers
# (respectively "cpu" and "cpuacct") with "-o cpuacct,cpu"
# but on a directory called "cpu,cpuacct" (note the inversion
# in the order of the groups). This tries to work around it.
[ $SUBSYS = cpuacct,cpu ] && ln -s $SUBSYS $CGROUP/cpu,cpuacct
done
# Note: as I write those lines, the LXC userland tools cannot setup
# a "sub-container" properly if the "devices" cgroup is not in its
# own hierarchy. Let's detect this and issue a warning.
grep -q :devices: /proc/1/cgroup ||
echo "WARNING: the 'devices' cgroup should be in its own hierarchy."
grep -qw devices /proc/1/cgroup ||
echo "WARNING: it looks like the 'devices' cgroup is not mounted."
# Now, close extraneous file descriptors.
pushd /proc/self/fd >/dev/null
for FD in *
do
case "$FD" in
# Keep stdin/stdout/stderr
[012])
;;
# Nuke everything else
*)
eval exec "$FD>&-"
;;
esac
done
popd >/dev/null
source /home/debian/.bash_profile
## at this point environment should be ready
## bridge to be used by gitian LXC container
brctl addbr br0 && \
ifconfig br0 ${GITIAN_HOST_IP}/16 up || exit $?
##NOTE: *DO NOT* try to add eth0 to the bridge, it will kill container's networking
## temporary workaround until this bug is fixed: https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/891045
umount /dev/shm
rmdir /dev/shm
ln -s /run/shm /dev/shm
##NOTE: this is setup here instead of Dockerfile because of a Docker glitch
AK=/root/authorized_keys
if [ -s $AK ]; then
cp $AK /home/debian/.ssh/ && \
rm $AK && \
chmod -R go-rwx /home/debian/.ssh &&
chown -R debian.debian /home/debian/.ssh || exit $?
fi
## test that debian user has access to its own .ssh (yes, Docker glitches crawling...)
su -c 'cat /home/debian/.ssh/authorized_keys' -l -- debian || exit $?
echo "Gitian host configuration for LXC guests completed successfully"

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

View File

@ -0,0 +1,446 @@
# This file lists locales that you wish to have built. You can find a list
# of valid supported locales at /usr/share/i18n/SUPPORTED, and you can add
# user defined locales to /usr/local/share/i18n/SUPPORTED. If you change
# this file, you need to rerun locale-gen.
# aa_DJ ISO-8859-1
# aa_DJ.UTF-8 UTF-8
# aa_ER UTF-8
# aa_ER@saaho UTF-8
# aa_ET UTF-8
# af_ZA ISO-8859-1
# af_ZA.UTF-8 UTF-8
# am_ET UTF-8
# an_ES ISO-8859-15
# an_ES.UTF-8 UTF-8
# ar_AE ISO-8859-6
# ar_AE.UTF-8 UTF-8
# ar_BH ISO-8859-6
# ar_BH.UTF-8 UTF-8
# ar_DZ ISO-8859-6
# ar_DZ.UTF-8 UTF-8
# ar_EG ISO-8859-6
# ar_EG.UTF-8 UTF-8
# ar_IN UTF-8
# ar_IQ ISO-8859-6
# ar_IQ.UTF-8 UTF-8
# ar_JO ISO-8859-6
# ar_JO.UTF-8 UTF-8
# ar_KW ISO-8859-6
# ar_KW.UTF-8 UTF-8
# ar_LB ISO-8859-6
# ar_LB.UTF-8 UTF-8
# ar_LY ISO-8859-6
# ar_LY.UTF-8 UTF-8
# ar_MA ISO-8859-6
# ar_MA.UTF-8 UTF-8
# ar_OM ISO-8859-6
# ar_OM.UTF-8 UTF-8
# ar_QA ISO-8859-6
# ar_QA.UTF-8 UTF-8
# ar_SA ISO-8859-6
# ar_SA.UTF-8 UTF-8
# ar_SD ISO-8859-6
# ar_SD.UTF-8 UTF-8
# ar_SY ISO-8859-6
# ar_SY.UTF-8 UTF-8
# ar_TN ISO-8859-6
# ar_TN.UTF-8 UTF-8
# ar_YE ISO-8859-6
# ar_YE.UTF-8 UTF-8
# as_IN.UTF-8 UTF-8
# ast_ES ISO-8859-15
# ast_ES.UTF-8 UTF-8
# az_AZ.UTF-8 UTF-8
# be_BY CP1251
# be_BY.UTF-8 UTF-8
# be_BY@latin UTF-8
# bem_ZM UTF-8
# ber_DZ UTF-8
# ber_MA UTF-8
# bg_BG CP1251
# bg_BG.UTF-8 UTF-8
# bn_BD UTF-8
# bn_IN UTF-8
# bo_CN UTF-8
# bo_IN UTF-8
# br_FR ISO-8859-1
# br_FR.UTF-8 UTF-8
# br_FR@euro ISO-8859-15
# bs_BA ISO-8859-2
# bs_BA.UTF-8 UTF-8
# byn_ER UTF-8
# ca_AD ISO-8859-15
# ca_AD.UTF-8 UTF-8
# ca_ES ISO-8859-1
# ca_ES.UTF-8 UTF-8
# ca_ES.UTF-8@valencia UTF-8
# ca_ES@euro ISO-8859-15
# ca_ES@valencia ISO-8859-15
# ca_FR ISO-8859-15
# ca_FR.UTF-8 UTF-8
# ca_IT ISO-8859-15
# ca_IT.UTF-8 UTF-8
# crh_UA UTF-8
# cs_CZ ISO-8859-2
# cs_CZ.UTF-8 UTF-8
# csb_PL UTF-8
# cv_RU UTF-8
# cy_GB ISO-8859-14
# cy_GB.UTF-8 UTF-8
# da_DK ISO-8859-1
# da_DK.UTF-8 UTF-8
# de_AT ISO-8859-1
# de_AT.UTF-8 UTF-8
# de_AT@euro ISO-8859-15
# de_BE ISO-8859-1
# de_BE.UTF-8 UTF-8
# de_BE@euro ISO-8859-15
# de_CH ISO-8859-1
# de_CH.UTF-8 UTF-8
# de_DE ISO-8859-1
# de_DE.UTF-8 UTF-8
# de_DE@euro ISO-8859-15
# de_LI.UTF-8 UTF-8
# de_LU ISO-8859-1
# de_LU.UTF-8 UTF-8
# de_LU@euro ISO-8859-15
# dv_MV UTF-8
# dz_BT UTF-8
# el_CY ISO-8859-7
# el_CY.UTF-8 UTF-8
# el_GR ISO-8859-7
# el_GR.UTF-8 UTF-8
# en_AG UTF-8
# en_AU ISO-8859-1
# en_AU.UTF-8 UTF-8
# en_BW ISO-8859-1
# en_BW.UTF-8 UTF-8
# en_CA ISO-8859-1
# en_CA.UTF-8 UTF-8
# en_DK ISO-8859-1
# en_DK.ISO-8859-15 ISO-8859-15
# en_DK.UTF-8 UTF-8
# en_GB ISO-8859-1
# en_GB.ISO-8859-15 ISO-8859-15
# en_GB.UTF-8 UTF-8
# en_HK ISO-8859-1
# en_HK.UTF-8 UTF-8
# en_IE ISO-8859-1
# en_IE.UTF-8 UTF-8
# en_IE@euro ISO-8859-15
# en_IN UTF-8
# en_NG UTF-8
# en_NZ ISO-8859-1
# en_NZ.UTF-8 UTF-8
# en_PH ISO-8859-1
# en_PH.UTF-8 UTF-8
# en_SG ISO-8859-1
# en_SG.UTF-8 UTF-8
# en_US ISO-8859-1
# en_US.ISO-8859-15 ISO-8859-15
en_US.UTF-8 UTF-8
# en_ZA ISO-8859-1
# en_ZA.UTF-8 UTF-8
# en_ZM UTF-8
# en_ZW ISO-8859-1
# en_ZW.UTF-8 UTF-8
# eo ISO-8859-3
# eo.UTF-8 UTF-8
# es_AR ISO-8859-1
# es_AR.UTF-8 UTF-8
# es_BO ISO-8859-1
# es_BO.UTF-8 UTF-8
# es_CL ISO-8859-1
# es_CL.UTF-8 UTF-8
# es_CO ISO-8859-1
# es_CO.UTF-8 UTF-8
# es_CR ISO-8859-1
# es_CR.UTF-8 UTF-8
# es_DO ISO-8859-1
# es_DO.UTF-8 UTF-8
# es_EC ISO-8859-1
# es_EC.UTF-8 UTF-8
# es_ES ISO-8859-1
# es_ES.UTF-8 UTF-8
# es_ES@euro ISO-8859-15
# es_GT ISO-8859-1
# es_GT.UTF-8 UTF-8
# es_HN ISO-8859-1
# es_HN.UTF-8 UTF-8
# es_MX ISO-8859-1
# es_MX.UTF-8 UTF-8
# es_NI ISO-8859-1
# es_NI.UTF-8 UTF-8
# es_PA ISO-8859-1
# es_PA.UTF-8 UTF-8
# es_PE ISO-8859-1
# es_PE.UTF-8 UTF-8
# es_PR ISO-8859-1
# es_PR.UTF-8 UTF-8
# es_PY ISO-8859-1
# es_PY.UTF-8 UTF-8
# es_SV ISO-8859-1
# es_SV.UTF-8 UTF-8
# es_US ISO-8859-1
# es_US.UTF-8 UTF-8
# es_UY ISO-8859-1
# es_UY.UTF-8 UTF-8
# es_VE ISO-8859-1
# es_VE.UTF-8 UTF-8
# et_EE ISO-8859-1
# et_EE.ISO-8859-15 ISO-8859-15
# et_EE.UTF-8 UTF-8
# eu_ES ISO-8859-1
# eu_ES.UTF-8 UTF-8
# eu_ES@euro ISO-8859-15
# eu_FR ISO-8859-1
# eu_FR.UTF-8 UTF-8
# eu_FR@euro ISO-8859-15
# fa_IR UTF-8
# ff_SN UTF-8
# fi_FI ISO-8859-1
# fi_FI.UTF-8 UTF-8
# fi_FI@euro ISO-8859-15
# fil_PH UTF-8
# fo_FO ISO-8859-1
# fo_FO.UTF-8 UTF-8
# fr_BE ISO-8859-1
# fr_BE.UTF-8 UTF-8
# fr_BE@euro ISO-8859-15
# fr_CA ISO-8859-1
# fr_CA.UTF-8 UTF-8
# fr_CH ISO-8859-1
# fr_CH.UTF-8 UTF-8
# fr_FR ISO-8859-1
# fr_FR.UTF-8 UTF-8
# fr_FR@euro ISO-8859-15
# fr_LU ISO-8859-1
# fr_LU.UTF-8 UTF-8
# fr_LU@euro ISO-8859-15
# fur_IT UTF-8
# fy_DE UTF-8
# fy_NL UTF-8
# ga_IE ISO-8859-1
# ga_IE.UTF-8 UTF-8
# ga_IE@euro ISO-8859-15
# gd_GB ISO-8859-15
# gd_GB.UTF-8 UTF-8
# gez_ER UTF-8
# gez_ER@abegede UTF-8
# gez_ET UTF-8
# gez_ET@abegede UTF-8
# gl_ES ISO-8859-1
# gl_ES.UTF-8 UTF-8
# gl_ES@euro ISO-8859-15
# gu_IN UTF-8
# gv_GB ISO-8859-1
# gv_GB.UTF-8 UTF-8
# ha_NG UTF-8
# he_IL ISO-8859-8
# he_IL.UTF-8 UTF-8
# hi_IN UTF-8
# hne_IN UTF-8
# hr_HR ISO-8859-2
# hr_HR.UTF-8 UTF-8
# hsb_DE ISO-8859-2
# hsb_DE.UTF-8 UTF-8
# ht_HT UTF-8
# hu_HU ISO-8859-2
# hu_HU.UTF-8 UTF-8
# hy_AM UTF-8
# hy_AM.ARMSCII-8 ARMSCII-8
# ia UTF-8
# id_ID ISO-8859-1
# id_ID.UTF-8 UTF-8
# ig_NG UTF-8
# ik_CA UTF-8
# is_IS ISO-8859-1
# is_IS.UTF-8 UTF-8
# it_CH ISO-8859-1
# it_CH.UTF-8 UTF-8
# it_IT ISO-8859-1
# it_IT.UTF-8 UTF-8
# it_IT@euro ISO-8859-15
# iu_CA UTF-8
# iw_IL ISO-8859-8
# iw_IL.UTF-8 UTF-8
# ja_JP.EUC-JP EUC-JP
# ja_JP.UTF-8 UTF-8
# ka_GE GEORGIAN-PS
# ka_GE.UTF-8 UTF-8
# kk_KZ PT154
# kk_KZ RK1048
# kk_KZ.UTF-8 UTF-8
# kl_GL ISO-8859-1
# kl_GL.UTF-8 UTF-8
# km_KH UTF-8
# kn_IN UTF-8
# ko_KR.EUC-KR EUC-KR
# ko_KR.UTF-8 UTF-8
# kok_IN UTF-8
# ks_IN UTF-8
# ks_IN@devanagari UTF-8
# ku_TR ISO-8859-9
# ku_TR.UTF-8 UTF-8
# kw_GB ISO-8859-1
# kw_GB.UTF-8 UTF-8
# ky_KG UTF-8
# lg_UG ISO-8859-10
# lg_UG.UTF-8 UTF-8
# li_BE UTF-8
# li_NL UTF-8
# lo_LA UTF-8
# lt_LT ISO-8859-13
# lt_LT.UTF-8 UTF-8
# lv_LV ISO-8859-13
# lv_LV.UTF-8 UTF-8
# mai_IN UTF-8
# mg_MG ISO-8859-15
# mg_MG.UTF-8 UTF-8
# mi_NZ ISO-8859-13
# mi_NZ.UTF-8 UTF-8
# mk_MK ISO-8859-5
# mk_MK.UTF-8 UTF-8
# ml_IN UTF-8
# mn_MN UTF-8
# mr_IN UTF-8
# ms_MY ISO-8859-1
# ms_MY.UTF-8 UTF-8
# mt_MT ISO-8859-3
# mt_MT.UTF-8 UTF-8
# my_MM UTF-8
# nan_TW@latin UTF-8
# nb_NO ISO-8859-1
# nb_NO.UTF-8 UTF-8
# nds_DE UTF-8
# nds_NL UTF-8
# ne_NP UTF-8
# nl_AW UTF-8
# nl_BE ISO-8859-1
# nl_BE.UTF-8 UTF-8
# nl_BE@euro ISO-8859-15
# nl_NL ISO-8859-1
# nl_NL.UTF-8 UTF-8
# nl_NL@euro ISO-8859-15
# nn_NO ISO-8859-1
# nn_NO.UTF-8 UTF-8
# nr_ZA UTF-8
# nso_ZA UTF-8
# oc_FR ISO-8859-1
# oc_FR.UTF-8 UTF-8
# om_ET UTF-8
# om_KE ISO-8859-1
# om_KE.UTF-8 UTF-8
# or_IN UTF-8
# os_RU UTF-8
# pa_IN UTF-8
# pa_PK UTF-8
# pap_AN UTF-8
# pl_PL ISO-8859-2
# pl_PL.UTF-8 UTF-8
# ps_AF UTF-8
# pt_BR ISO-8859-1
# pt_BR.UTF-8 UTF-8
# pt_PT ISO-8859-1
# pt_PT.UTF-8 UTF-8
# pt_PT@euro ISO-8859-15
# ro_RO ISO-8859-2
# ro_RO.UTF-8 UTF-8
# ru_RU ISO-8859-5
# ru_RU.CP1251 CP1251
# ru_RU.KOI8-R KOI8-R
# ru_RU.UTF-8 UTF-8
# ru_UA KOI8-U
# ru_UA.UTF-8 UTF-8
# rw_RW UTF-8
# sa_IN UTF-8
# sc_IT UTF-8
# sd_IN UTF-8
# sd_IN@devanagari UTF-8
# se_NO UTF-8
# shs_CA UTF-8
# si_LK UTF-8
# sid_ET UTF-8
# sk_SK ISO-8859-2
# sk_SK.UTF-8 UTF-8
# sl_SI ISO-8859-2
# sl_SI.UTF-8 UTF-8
# so_DJ ISO-8859-1
# so_DJ.UTF-8 UTF-8
# so_ET UTF-8
# so_KE ISO-8859-1
# so_KE.UTF-8 UTF-8
# so_SO ISO-8859-1
# so_SO.UTF-8 UTF-8
# sq_AL ISO-8859-1
# sq_AL.UTF-8 UTF-8
# sq_MK UTF-8
# sr_ME UTF-8
# sr_RS UTF-8
# sr_RS@latin UTF-8
# ss_ZA UTF-8
# st_ZA ISO-8859-1
# st_ZA.UTF-8 UTF-8
# sv_FI ISO-8859-1
# sv_FI.UTF-8 UTF-8
# sv_FI@euro ISO-8859-15
# sv_SE ISO-8859-1
# sv_SE.ISO-8859-15 ISO-8859-15
# sv_SE.UTF-8 UTF-8
# sw_KE UTF-8
# sw_TZ UTF-8
# ta_IN UTF-8
# te_IN UTF-8
# tg_TJ KOI8-T
# tg_TJ.UTF-8 UTF-8
# th_TH TIS-620
# th_TH.UTF-8 UTF-8
# ti_ER UTF-8
# ti_ET UTF-8
# tig_ER UTF-8
# tk_TM UTF-8
# tl_PH ISO-8859-1
# tl_PH.UTF-8 UTF-8
# tn_ZA UTF-8
# tr_CY ISO-8859-9
# tr_CY.UTF-8 UTF-8
# tr_TR ISO-8859-9
# tr_TR.UTF-8 UTF-8
# ts_ZA UTF-8
# tt_RU.UTF-8 UTF-8
# tt_RU.UTF-8@iqtelif UTF-8
# ug_CN UTF-8
# uk_UA KOI8-U
# uk_UA.UTF-8 UTF-8
# ur_PK UTF-8
# uz_UZ ISO-8859-1
# uz_UZ.UTF-8 UTF-8
# uz_UZ@cyrillic UTF-8
# ve_ZA UTF-8
# vi_VN UTF-8
# vi_VN.TCVN TCVN5712-1
# wa_BE ISO-8859-1
# wa_BE.UTF-8 UTF-8
# wa_BE@euro ISO-8859-15
# wo_SN UTF-8
# xh_ZA ISO-8859-1
# xh_ZA.UTF-8 UTF-8
# yi_US CP1255
# yi_US.UTF-8 UTF-8
# yo_NG UTF-8
# zh_CN GB2312
# zh_CN.GB18030 GB18030
# zh_CN.GBK GBK
# zh_CN.UTF-8 UTF-8
# zh_HK BIG5-HKSCS
# zh_HK.UTF-8 UTF-8
# zh_SG GB2312
# zh_SG.GBK GBK
# zh_SG.UTF-8 UTF-8
# zh_TW BIG5
# zh_TW.EUC-TW EUC-TW
# zh_TW.UTF-8 UTF-8
# zu_ZA ISO-8859-1
# zu_ZA.UTF-8 UTF-8

View File

@ -0,0 +1 @@
Europe/Amsterdam

View File

@ -0,0 +1,22 @@
diff --git a/bin/make-base-vm b/bin/make-base-vm
index a7c182b..8dc4091 100755
--- a/bin/make-base-vm
+++ b/bin/make-base-vm
@@ -227,5 +227,17 @@ else
mv $OUT/*.qcow2 $OUT.qcow2
rm -rf $OUT
# bootstrap-fixup is done on first boot
+
+ if [ $LXC = "1" ]; then
+ #sudo debootstrap --include=$addpkg --arch=$ARCH $SUITE $OUT-root $MIRROR
+ echo Extracting partition for lxc
+ qemu-img convert $OUT.qcow2 $OUT.raw
+ loop=`sudo kpartx -av $OUT.raw|sed -n '/loop.p1/{s/.*loop\(.\)p1.*/\1/;p}'`
+ sudo cp --sparse=always /dev/mapper/loop${loop}p1 $OUT
+ sudo chown $USER $OUT
+ sudo kpartx -d /dev/loop$loop
+ rm -f $OUT.raw
+ # bootstrap-fixup is done in libexec/make-clean-vm
+ fi
fi

Binary file not shown.

Binary file not shown.

View File

@ -0,0 +1,179 @@
#!/bin/bash
## bitcoin-gitian-build.sh
##
## @author gdm85
##
## Automatically build latest version of Bitcoin Core using
## Docker containers (nested LXC).
##
## User can specify target operative systems as arguments.
## Several optional environment variables condition the build:
## - OUTPUTDIR - where input/output volume directories will be read/created
## - SIGNER - id of signer (no signature will be attempted, just directory structure created)
## - COMMIT - commit/branch to use for build, by default is latest tag
## - NOPURGE - set to non-empty to not dispose containers after build
#
SCRIPTS=$(dirname $(readlink -m $0)) || exit $?
## place this file in script's directory in order to build for Mac OS X
SDK=MacOSX10.7.sdk.tar.gz
if [ $# -lt 1 ]; then
echo "Usage: gitian-build.sh linux [win] [osx] [...]" 1>&2
exit 1
fi
if docker info 2>/dev/null | grep ^Storage | grep aufs$ >/dev/null; then
echo "You are using AUFS as Docker storage drive, which is terribly broken and not supported by this script." 1>&2
exit 1
fi
## identify a CLI tool to run commands in parallel
## coshell is preferred
PARALLEL=""
if type coshell 2>/dev/null >/dev/null; then
PARALLEL="coshell"
else
if type parallel 2>/dev/null >/dev/null; then
PARALLEL="parallel -j$#"
else
echo "Please install coshell (https://github.com/gdm85/coshell) or GNU Parallel (https://www.gnu.org/software/parallel/)" 1>&2
exit 2
fi
fi
function read_commit() {
local SHA="$1"
local OUTPUT
set -o pipefail && \
OUTPUT=$(curl -s https://api.github.com/repos/bitcoin/bitcoin/commits/${SHA} | jq -r '.sha') && \
test ! -z "$OUTPUT" && \
test "$OUTPUT" != "null" && \
echo "$OUTPUT"
}
## run all necessary containers, detached
## setup proper volumes for input/output collection
function run_all() {
local OS
for OS in "$@"; do
mkdir -p "$LSOURCE/${OS}" && \
rm -rf "$LDEST/${OS}" && \
mkdir -p "$LDEST/${OS}" || return $?
done
mkdir -p "$LSIGS" && \
mkdir -p "$LSOURCE" && \
mkdir -p "$LRESULT" && \
chown -R 1000.1000 "$LDEST" "$LSOURCE" "$LSIGS" "$LRESULT" || return $?
for OS in "$@"; do
echo "docker run -d --privileged -v $LRESULT:$CRESULT -v $LSIGS:$CSIGS -v $LSOURCE/${OS}:${CSOURCE} -v $LDEST/${OS}:$CDEST gdm85/gitian-bitcoin-host" || return $?
done | $PARALLEL
}
function inject_mac_sdk() {
local CID="$1"
docker-inject "$CID" "$SCRIPTS/$SDK" /home/debian/gitian-builder/inputs/
}
function build_all() {
local ALL=($@)
local COUNT=$#
local LEN=$((COUNT/2))
local CREATED=(${ALL[@]:0:$LEN})
local OSES=(${ALL[@]:$LEN})
local CID
local OS
local I=0
for CID in "${CREATED[@]}"; do
OS=${OSES[$I]}
if [[ "$OS" == "osx" ]]; then
inject_mac_sdk "$CID" || return $?
fi
let I+=1
done
I=0
for CID in "${CREATED[@]}"; do
OS=${OSES[$I]}
local OS_LOG_FILE="$LLOGS/build-${OS}.log"
echo "Execution log for ${OS} ({$HCOMMIT}) --> $OS_LOG_FILE" 1>&2
echo -n "docker exec $CID su -c 'cd /home/debian && source .bash_profile && ./build-bitcoin.sh $COMMIT ${OS} && " && \
echo -n "cd gitian-builder && ./bin/gasserts --signer $SIGNER --release ${HCOMMIT} --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-${OS}.yml' debian " && \
echo " >> $OS_LOG_FILE 2>&1"
let I+=1
done | $PARALLEL
}
## change the assert directory as desired
if [ -z "$SIGNER" ]; then
SIGNER="$USER"
fi
## customize output volumes
if [ -z "$OUTPUTDIR" ]; then
OUTPUTDIR="$SCRIPTS/output"
fi
set -o pipefail || exit $?
## always get latest release/rc if no commit environment was specified
if [ ! -z "$COMMIT" ]; then
HCOMMIT="$COMMIT"
else
HCOMMIT="$(curl -s https://api.github.com/repos/bitcoin/bitcoin/tags | jq -r '.[0].name')" || exit $?
fi
## get commit short hash
## NOTE: this overwrites environment provided by user
COMMIT=$(read_commit "$HCOMMIT") || exit $?
###
### declarations for input/output data volumes
###
## always add human readable commit and commit to volume path variables
REL_OD="$OUTPUTDIR/${HCOMMIT}-${COMMIT}"
LRESULT="${REL_OD}/result-${HCOMMIT}-${COMMIT}"
LSIGS="${REL_OD}/sigs"
LDEST="${REL_OD}/built"
LLOGS="${REL_OD}"
## depends-cache does not sport human readable prefix, being the only input volume for containers
LSOURCE="${OUTPUTDIR}/${COMMIT}/depends-cache"
## path of above volumes inside the containers
CRESULT="/home/debian/gitian-builder/result"
CSIGS="/home/debian/gitian.sigs"
CSOURCE="/home/debian/gitian-builder/cache"
CDEST="/home/debian/gitian-builder/build"
## ---------------- main -------------------- ##
CREATED="$(run_all $@ | tr '\n' ' ')" && \
echo "Building bitcoin (${HCOMMIT}) for $@" && \
build_all ${CREATED[@]} $@
RV=$?
if [ -z "$NOPURGE" ]; then
## cleanup
#echo "Cleaning up created containers..."
for CID in $CREATED; do
docker stop $CID
docker rm $CID
done
fi
## return build exit code
if [ $RV -eq 0 ]; then
echo -n "Completed successfully "
else
echo -n "Failed "
fi
echo "with exit code = $RV"
exit $RV

View File

@ -0,0 +1,29 @@
#!/bin/bash
## build-macos121-wily-kernel.sh
##
## @author gdm85
##
## Build a kernel with fixed bluetooth support for Mac OS 12,1 and Ubuntu Wily
## Based on instructions read from http://www.spinics.net/lists/linux-bluetooth/msg64123.html
##
#
SCRIPTS=$(dirname $(readlink -m $0)) || exit $?
set -e
cd "$SCRIPTS"
if ! docker inspect gdm85/wily >/dev/null 2>/dev/null; then
./build-ubuntu-image.sh wily
fi
cd ../ubuntu-pkgbuilder
make wily
cd ../ubuntu-kernelbuilder
make wily linux-image-wily
echo "Linux kernel .deb packages are now available in packages/"

View File

@ -0,0 +1,72 @@
#!/bin/bash
## @author gdm85
##
## build a base Ubuntu image, can be Trusty, Wily etc
#
BASENAME=$(dirname $(readlink -m $0))
## the distro we are going to use
REPOSRC=http://archive.ubuntu.com/ubuntu/
if [ ! $UID -eq 0 ]; then
echo "This script can only be run as root" 1>&2
exit 1
fi
if [ ! $# -eq 1 ]; then
echo "Usage: build-ubuntu-image.sh release-name" 1>&2
exit 2
fi
DISTNAME="$1"
## check for prerequisites
if ! type -P debootstrap >/dev/null; then
echo "You need to install debootstrap" 1>&2
exit 2
fi
## check about the Ubuntu archive keyring
DEFK=/usr/share/keyrings/ubuntu-archive-keyring.gpg
KEYRING=$BASENAME/../keyrings/ubuntu-archive-keyring.gpg
if [ -s $DEFK ]; then
if ! diff $DEFK $KEYRING; then
ANSWER=
while [[ "$ANSWER" != "Y" && "$ANSWER" != "n" ]]; do
echo -n "The Ubuntu Archive keyring in your system ($DEFK) that will be used to debootstrap is different from the reference provided keyring. Continue? (Y/n) "
read -r ANSWER || exit $?
done
if [[ "$ANSWER" == "n" ]]; then
exit 1
fi
## use system's keyring, even if different than provided one
## this is a no-issue only in case the system's keyring is more recent than the provided one
KEYRING=$DEFK
fi
else
ANSWER=
while [[ "$ANSWER" != "Y" && "$ANSWER" != "n" ]]; do
echo -n "Your system comes with no Ubuntu Archive keyring in $DEFK that is necessary for debootstrap. Use reference provided keyring? (Y/n) "
read -r ANSWER || exit $?
done
if [[ "$ANSWER" == "n" ]]; then
exit 1
fi
fi
echo "Will use $KEYRING"
## NOTE: a temporary directory under /tmp is not used because can't be mounted dev/exec
mkdir $DISTNAME || exit $?
TMPDIR=$PWD/$DISTNAME
debootstrap --keyring=$KEYRING $DISTNAME $DISTNAME $REPOSRC && \
cd $DISTNAME && \
tar -c . | docker import - gdm85/$DISTNAME
RV=$?
# always perform cleanup
rm -rf $TMPDIR
exit $RV

66
docker/scripts/build-wheezy.sh Executable file
View File

@ -0,0 +1,66 @@
#!/bin/bash
## @author gdm85
##
## build a base Debian Wheezy
#
BASENAME=$(dirname $(readlink -m $0))
## the distro we are going to use
DISTNAME=wheezy
DEBIAN_REPO=http://ftp.debian.org/debian
if [ ! $UID -eq 0 ]; then
echo "This script can only be run as root" 1>&2
exit 1
fi
## install prerequisites
## NOTE: may fail on non-Ubuntu/Debian systems
if ! type -P debootstrap >/dev/null; then
apt-get install debootstrap -y || exit $?
fi
## check about the Debian archive keyring
DEFK=/usr/share/keyrings/debian-archive-keyring.gpg
KEYRING=$BASENAME/../keyrings/debian-archive-keyring.gpg
if [ -s $DEFK ]; then
if ! diff $DEFK $KEYRING; then
ANSWER=
while [[ "$ANSWER" != "Y" && "$ANSWER" != "n" ]]; do
echo -n "The Debian Archive keyring in your system ($DEFK) that will be used to debootstrap is different from the reference provided keyring. Continue? (Y/n) "
read -r ANSWER || exit $?
done
if [[ "$ANSWER" == "n" ]]; then
exit 1
fi
## use system's keyring, even if different than provided one
## this is a no-issue only in case the system's keyring is more recent than the provided one
KEYRING=$DEFK
fi
else
ANSWER=
while [[ "$ANSWER" != "Y" && "$ANSWER" != "n" ]]; do
echo -n "Your system comes with no Debian Archive keyring in $DEFK that is necessary for debootstrap. Use reference provided keyring? (Y/n) "
read -r ANSWER || exit $?
done
if [[ "$ANSWER" == "n" ]]; then
exit 1
fi
fi
echo "Will use $KEYRING"
## NOTE: a temporary directory under /tmp is not used because can't be mounted dev/exec
mkdir $DISTNAME || exit $?
TMPDIR=$PWD/$DISTNAME
debootstrap --keyring=$KEYRING $DISTNAME $DISTNAME $DEBIAN_REPO && \
cd $DISTNAME && \
tar -c . | docker import - gdm85/$DISTNAME
RV=$?
# always perform cleanup
rm -rf $TMPDIR
exit $RV

View File

@ -0,0 +1,8 @@
#!/bin/bash
set -e
BASENAME=$(dirname $(readlink -m $0))
cd $BASENAME/../gitian-bitcoin-host && \
docker build --tag=gdm85/gitian-bitcoin-host .

View File

@ -0,0 +1,52 @@
#!/bin/bash
BASENAME=$(dirname $(readlink -m $0))
function wait_for_ssh() {
local IP="$1"
local SECS="$2"
while [ $SECS -gt 0 ]; do
ssh -o ConnectTimeout=1 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@$IP ls >/dev/null 2>/dev/null && return 0
sleep 1
let SECS-=1
done
return 1
}
function wait_remove() {
local CID="$1"
while [ ! docker rm $CID 2>/dev/null ]; do
sleep 2
done
}
cd $BASENAME/../gitian-host || exit $?
if [ ! -f authorized_keys ]; then
echo "No authorized_keys file found in $PWD"
if [ -f ~/.ssh/id_rsa.pub ]; then
echo -n "Do you want to use ~/.ssh/id_rsa.pub? (y/n) "
read -r ANSWER
if [[ "$ANSWER" == "y" ]]; then
cp -v ~/.ssh/id_rsa.pub authorized_keys || exit $?
else
exit 1
fi
else
exit 1
fi
fi
##NOTE: can leave behind a running container of gitian-host
docker build --tag=gdm85/gitian-host . && \
CID=$(docker run -d --privileged gdm85/gitian-host) && \
IP=$(docker exec $CID cat /etc/hosts | tail -n1 | awk '{ print $1 }') && \
wait_for_ssh "$IP" 10 && \
echo "$CID is now online ($IP), building base VMs on it" && \
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@$IP bash -c 'cd /home/debian && source ./.bash_profile && ./build-base-vms.sh amd64' && \
docker kill $CID && \
docker wait $CID && \
docker commit $CID gdm85/gitian-host-vms && \
wait_remove $CID && \
echo "Gitian host images created successfully!" && \
echo "You can now spawn containers with spawn-gitian-host.sh"

View File

@ -0,0 +1,9 @@
#!/bin/bash
set -e
CID=$(docker run -d --privileged gdm85/gitian-bitcoin-host) || exit $?
IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $CID) || exit $?
echo "You can now SSH into container $CID:"
echo "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@$IP"

View File

@ -0,0 +1,9 @@
#!/bin/bash
set -e
CID=$(docker run -d --privileged gdm85/gitian-host-vms) && \
IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' $CID) || exit $?
echo "You can now SSH into container $CID:"
echo "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no debian@$IP"

View File

@ -0,0 +1,16 @@
FROM %IMAGE%-pkgbuilder
ENV KERNEL_VERSION %KERNEL_VERSION%
RUN apt-get build-dep -y --no-install-recommends linux-image-$KERNEL_VERSION && \
apt-get install -y git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev
USER rdeckard
## fetch source
RUN mkdir build && cd build && apt-get source linux-image-$KERNEL_VERSION
## add build script
COPY build-kernel.sh /home/rdeckard/
RUN chown rdeckard.rdeckard /home/rdeckard/build-kernel.sh

View File

@ -0,0 +1,15 @@
ifeq ($(KERNEL_VERSION),)
KERNEL_VERSION := $(shell uname -r)
endif
linux-image-%:
mkdir -p packages
docker run -e KERNEL_VERSION=$(KERNEL_VERSION) -v $(CURDIR)/packages:/home/rdeckard/packages -v $(CURDIR)/patches:/home/rdeckard/patches:ro $*-kernelbuilder ./build-kernel.sh $(KERNEL_VERSION)
## supports any Ubuntu version (trusty, wily etc)
%:
TMPD=`mktemp -d` && cp build-kernel.sh "$$TMPD/" && \
sed -e 's~%IMAGE%~$*~g' -e 's~%KERNEL_VERSION%~$(KERNEL_VERSION)~' Dockerfile.tpl > "$$TMPD/Dockerfile" && cd "$$TMPD" && \
docker build --tag='$*-kernelbuilder' . && cd && rm -rf "$TMPD"
.DEFAULT: trusty

View File

@ -0,0 +1,17 @@
#!/bin/bash
set -e
## enter the source-unpacked location
cd build/linux-*
## will fail here if no patches are available
for MYP in $(ls ../../patches); do
patch -p1 < ../../patches/$MYP
done
fakeroot debian/rules clean
DEB_BUILD_OPTIONS=parallel=4 AUTOBUILD=1 NOEXTRAS=1 fakeroot debian/rules binary-generic
mv ../*.deb $HOME/packages/

View File

@ -0,0 +1,44 @@
--- a/drivers/bluetooth/btusb.c 2015-11-24 09:53:01.230539986 +0000
+++ b/drivers/bluetooth/btusb.c 2015-11-24 09:54:32.598314339 +0000
@@ -38,6 +38,9 @@
static bool disable_scofix;
static bool force_scofix;
+static int probe_interface = 2;
+static int usb_interface = 3;
+
static bool reset = true;
static struct usb_driver btusb_driver;
@@ -2657,7 +2660,7 @@
BT_DBG("intf %p id %p", intf, id);
/* interface numbers are hardcoded in the spec */
- if (intf->cur_altsetting->desc.bInterfaceNumber != 2)
+ if (intf->cur_altsetting->desc.bInterfaceNumber != probe_interface)
return -ENODEV;
if (!id->driver_info) {
@@ -2827,7 +2830,7 @@
data->isoc = NULL;
} else {
/* Interface numbers are hardcoded in the specification */
- data->isoc = usb_ifnum_to_if(data->udev, 3);
+ data->isoc = usb_ifnum_to_if(data->udev, usb_interface);
}
if (!reset)
@@ -3057,6 +3060,12 @@
module_param(force_scofix, bool, 0644);
MODULE_PARM_DESC(force_scofix, "Force fixup of wrong SCO buffers size");
+module_param(probe_interface, int, 0644);
+MODULE_PARM_DESC(probe_interface, "Interface to probe");
+
+module_param(usb_interface, int, 0644);
+MODULE_PARM_DESC(probe_interface, "USB interface to use");
+
module_param(reset, bool, 0644);
MODULE_PARM_DESC(reset, "Send HCI reset command on initialization");

View File

@ -0,0 +1,27 @@
## ubuntu-pkgbuilder
##
## VERSION 0.1.1
##
## Ubuntu image to build a .deb package
##
#
FROM %IMAGE%
MAINTAINER Giuseppe Mazzotta "gdm85@users.noreply.github.com"
ENV DEBIAN_FRONTEND noninteractive
## replace sources
COPY sources.list /etc/apt/sources.list
RUN apt-get update && apt-get install -y apt-utils aptitude && aptitude update && aptitude safe-upgrade -y && \
aptitude install -y nano tmux fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge
## log rotation is not managed in this container, thus remove it
RUN apt-get remove -y logrotate
## user that will make the compilation
RUN useradd -m -s /bin/bash rdeckard && mkdir /home/rdeckard/patches && chown rdeckard.rdeckard /home/rdeckard/patches
WORKDIR /home/rdeckard

View File

@ -0,0 +1,7 @@
.DEFAULT: trusty
## supports any base Ubuntu image (gdm85/trusty, gdm85/wily etc)
## they should be generated with make-ubuntu-image.sh
%:
TMPD=`mktemp -d` && sed 's~%IMAGE%~gdm85/$@~g' Dockerfile.tpl > "$$TMPD/Dockerfile" && sed 's~%DIST%~$@~g' sources.list.tpl > "$$TMPD/sources.list" && cd "$$TMPD" && \
docker build --tag='$@-pkgbuilder' . && cd && rm -rf "$TMPD"

View File

@ -0,0 +1,11 @@
deb http://archive.ubuntu.com/ubuntu/ %DIST% main restricted
deb-src http://archive.ubuntu.com/ubuntu/ %DIST% main restricted
deb http://archive.ubuntu.com/ubuntu/ %DIST%-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ %DIST%-updates main restricted
deb http://security.ubuntu.com/ubuntu %DIST%-security main restricted
deb-src http://security.ubuntu.com/ubuntu %DIST%-security main restricted
deb http://security.ubuntu.com/ubuntu %DIST%-security universe
deb-src http://security.ubuntu.com/ubuntu %DIST%-security universe
deb http://security.ubuntu.com/ubuntu %DIST%-security multiverse
deb-src http://security.ubuntu.com/ubuntu %DIST%-security multiverse

25
lxd/README.md Normal file
View File

@ -0,0 +1,25 @@
# lxd-images
Set of images for use with [LXD](https://linuxcontainers.org/lxd/).
<a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-sa/2.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/">Creative Commons Attribution-ShareAlike 2.0 Generic License</a>.
## busybox-nonroot
Running the containers' processes as non-root with ``busybox-nonroot`` is useful when you design applications that do not need root privileges,
as you will be able to detect problems that would otherwise be unnoticed when running with root privileges.
Create the image tarball with:
```
make
```
This will import the standard ``busybox`` image if you don't already have it.
Afterwards you can import ``busybox-nonroot`` image with:
```
make import
```
This image has a ``nobody`` user defined, so that you can run all processes inside the container as a non-privileged user.
**NOTE:** this is fundamentally different from root uid/gid mapping (which you should still use, if possible), as explained in [this blog post](https://www.stgraber.org/2014/01/17/lxc-1-0-unprivileged-containers/).

View File

@ -0,0 +1,19 @@
.DEFAULT := all
.PHONY := all import have-busybox
## current hash of busybox image
BASEIMG := 6e9df9219c50317368ec1e2b37da8bb1f5edcbefae3bae4bd38f41b56a365d09
all: busybox-nonroot.tar.xz
have-busybox:
lxc image info busybox >/dev/null || lxd-images import busybox --alias busybox
busybox-nonroot.tar.xz: have-busybox
TMPD=`mktemp -d` && mkdir -p "$$TMPD/rootfs/etc" && \
cp overlay/etc/passwd overlay/etc/group "$$TMPD/rootfs/etc/" && cd "$$TMPD" && tar xf /var/lib/lxd/images/$(BASEIMG) && \
chmod go+w /tmp && chmod +x /tmp && \
tar pcfJ $(CURDIR)/$@ rootfs/ metadata.yaml && cd && rm -rf "$$TMPD"
import:
lxc image info busybox-nonroot >/dev/null 2>/dev/null || lxc image import busybox-nonroot.tar.xz --alias busybox-nonroot

View File

@ -0,0 +1,11 @@
{
"architecture": "x86_64",
"creation_date": 1443204631,
"properties": {
"architecture": "x86_64",
"description": "Busybox with unprivileged user x86_64",
"name": "busybox-nonroot-x86_64",
"author": "gdm85",
"os": "Busybox"
}
}

View File

@ -0,0 +1 @@
nobody:x:1:

View File

@ -0,0 +1 @@
nobody:x:1:1:nobody:/tmp:/bin/sh

1
rkt/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
nginx/nginx.aci

22
rkt/Makefile Normal file
View File

@ -0,0 +1,22 @@
## define here all recursively buildable targets
SUBDIRS := nginx
.PHONY := $(SUBDIRS) recurse clean get-tools
.DEFAULT_GOAL := recurse
RECURSE_TARGET ?= $(MAKECMDGOALS)
recurse:
@for D in $(SUBDIRS) ; do \
if [ -e $${D}/Makefile ]; then \
$(MAKE) -C "$${D}" $(RECURSE_TARGET) || exit $$? ; \
fi ; \
done
clean: ; @$(MAKE) RECURSE_TARGET=clean recurse
## disable entering/exiting messages
#.SILENT:
get-tools:
go get github.com/gdm85/deb2aci

28
rkt/README.md Normal file
View File

@ -0,0 +1,28 @@
rkt images
==========
This repository contains recipes for various [ACI](https://github.com/appc/spec/blob/master/spec/aci.md) images that can be used with [rkt](https://github.com/coreos/rkt).
<a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/"><img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by-sa/2.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/2.0/">Creative Commons Attribution-ShareAlike 2.0 Generic License</a>.
Building
========
You will need [deb2aci](https://github.com/gdm85/deb2aci) to build these ACI images; you can fetch it automatically into your ``GOPATH`` with:
```
make get-tools
```
To build all images, run:
```
make
```
To build a specific image, for example ``nginx``, run:
```
make -C nginx
```
**DO NOT** run ``make nginx``, it would not build anything.
Each image subdirectory comes with a README.md for specific documentation and instructions.

13
rkt/nginx/Makefile Normal file
View File

@ -0,0 +1,13 @@
.DEFAULT_GOAL := all
.PHONY := all clean
## all necessary deb packages
DEBS := nginx
all: nginx.aci
nginx.aci:
deb2aci -pkg $(DEBS) -manifest build.manifest -image $@
clean:
rm -f nginx.aci

38
rkt/nginx/build.manifest Normal file
View File

@ -0,0 +1,38 @@
{
"acKind": "ImageManifest",
"acVersion": "0.6.1",
"name": "gdm85/rkt-images/nginx",
"app":{
"user": "65534",
"group": "65534",
"exec":[
"/usr/sbin/nginx"
],
"mountPoints": [
{
"name": "etc",
"path": "/etc/nginx",
"readOnly": true
},
{
"name": "www",
"path": "/var/www",
"readOnly": true
}
],
"ports": [
{
"name": "www",
"port": 80,
"protocol": "tcp",
"socketActivated": false
}
]
},
"annotations":[
{
"name":"authors",
"value":"gdm85 <gdm85@users.noreply.github.com>"
}
]
}