algo/docs/deploy-from-ansible.md

283 lines
9.9 KiB
Markdown
Raw Normal View History

# Deployment from Ansible
2016-10-13 13:27:06 +00:00
Before you begin, make sure you have installed all the dependencies necessary for your operating system as described in the [README](../README.md).
2016-10-13 13:27:06 +00:00
You can deploy Algo non-interactively by running the Ansible playbooks directly with `ansible-playbook`.
`ansible-playbook` accepts variables via the `-e` or `--extra-vars` option. You can pass variables as space separated key=value pairs. Algo requires certain variables that are listed below. You can also use the `--skip-tags` option to skip certain parts of the install, such as `iptables` (overwrite iptables rules), `ipsec` (install strongSwan), `wireguard` (install Wireguard). We don't recommend using the `-t` option as it will only include the tagged portions of the deployment, and skip certain necessary roles (such as `common`).
Here is a full example for DigitalOcean:
2016-10-13 13:27:06 +00:00
```shell
ansible-playbook main.yml -e "provider=digitalocean
server_name=algo
ondemand_cellular=false
ondemand_wifi=false
dns_adblocking=true
ssh_tunneling=true
store_pki=true
region=ams3
do_token=token"
2016-10-13 13:27:06 +00:00
```
See below for more information about variables and roles.
### Variables
2016-10-13 13:27:06 +00:00
- `provider` - (Required) The provider to use. See possible values below
- `server_name` - (Required) Server name. Default: algo
- `ondemand_cellular` (Optional) Enables VPN On Demand when connected to cellular networks for iOS/macOS clients using IPsec. Default: false
- `ondemand_wifi` - (Optional. See `ondemand_wifi_exclude`) Enables VPN On Demand when connected to WiFi networks for iOS/macOS clients using IPsec. Default: false
- `ondemand_wifi_exclude` (Required if `ondemand_wifi` set) - WiFi networks to exclude from using the VPN. Comma-separated values
- `dns_adblocking` - (Optional) Enables dnscrypt-proxy adblocking. Default: false
- `ssh_tunneling` - (Optional) Enable SSH tunneling for each user. Default: false
- `store_pki` - (Optional) Whether or not keep the CA key (required to add users in the future, but less secure). Default: false
2017-01-04 14:16:55 +00:00
If any of the above variables are unspecified, ansible will ask the user to input them.
### Ansible roles
Cloud roles can be activated by specifying an extra variable `provider`.
2017-01-04 14:16:55 +00:00
2016-10-13 13:27:06 +00:00
Cloud roles:
Refactor to support Ansible 2.8 (#1549) * bump ansible to 2.8.3 * DigitalOcean: move to the latest modules * Add Hetzner Cloud * Scaleway and Lightsail fixes * lint missing roles * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Try to run apt until succeeded * Scaleway modules upgrade * GCP: Refactoring, remove deprecated modules * Doc updates (#1552) * Update README.md Adding links and mentions of Exoscale aka CloudStack and Hetzner Cloud. * Update index.md Add the Hetzner Cloud to the docs index * Remove link to Win 10 IPsec instructions * Delete client-windows.md Unnecessary since the deprecation of IPsec for Win10. * Update deploy-from-ansible.md Added sections and required variables for CloudStack and Hetzner Cloud. * Update deploy-from-ansible.md Added sections for CloudStack and Hetzner, added req variables and examples, mentioned environment variables, and added links to the provider role section. * Update deploy-from-ansible.md Cosmetic changes to links, fix typo. * Update GCE variables * Update deploy-from-script-or-cloud-init-to-localhost.md Fix a finer point, and make variables list more readable. * update azure requirements * Python3 draft * set LANG=c to the p12 password generation task * Update README * Install cloud requirements to the existing venv * FreeBSD fix * env->.env fixes * lightsail_region_facts fix * yaml syntax fix * Update README for Python 3 (#1564) * Update README for Python 3 * Remove tabs and tweak instructions * Remove cosmetic command indentation * Update README.md * Update README for Python 3 (#1565) * DO fix for "found unpermitted parameters: id" * Verify Python version * Remove ubuntu 16.04 from readme * Revert back DigitalOcean module * Update deploy-from-script-or-cloud-init-to-localhost.md * env to .env
2019-09-28 00:10:20 +00:00
- role: cloud-digitalocean, [provider: digitalocean](#digital-ocean)
- role: cloud-ec2, [provider: ec2](#amazon-ec2)
- role: cloud-gce, [provider: gce](#google-compute-engine)
- role: cloud-vultr, [provider: vultr](#vultr)
- role: cloud-azure, [provider: azure](#azure)
- role: cloud-lightsail, [provider: lightsail](#lightsail)
- role: cloud-scaleway, [provider: scaleway](#scaleway)
- role: cloud-openstack, [provider: openstack](#openstack)
- role: cloud-cloudstack, [provider: cloudstack](#cloudstack)
- role: cloud-hetzner, [provider: hetzner](#hetzner)
2016-10-13 13:27:06 +00:00
Server roles:
- role: strongswan
* Installs [strongSwan](https://www.strongswan.org/)
* Enables AppArmor, limits CPU and memory access, and drops user privileges
* Builds a Certificate Authority (CA) with [easy-rsa-ipsec](https://github.com/ValdikSS/easy-rsa-ipsec) and creates one client certificate per user
* Bundles the appropriate certificates into Apple mobileconfig profiles for each user
- role: dns_adblocking
* Installs DNS encryption through [dnscrypt-proxy](https://github.com/jedisct1/dnscrypt-proxy) with blacklists to be updated daily from `adblock_lists` in `config.cfg` - note this will occur even if `dns_encryption` in `config.cfg` is set to `false`
* Constrains dnscrypt-proxy with AppArmor and cgroups CPU and memory limitations
- role: ssh_tunneling
* Adds a restricted `algo` group with no shell access and limited SSH forwarding options
* Creates one limited, local account and an SSH public key for each user
- role: wireguard
* Installs a [Wireguard](https://www.wireguard.com/) server, with a startup script, and automatic checks for upgrades
* Creates wireguard.conf files for Linux clients as well as QR codes for Apple/Android clients
2016-10-13 13:27:06 +00:00
Note: The `strongswan` role generates Apple profiles with On-Demand Wifi and Cellular if you pass the following variables:
- ondemand_wifi: true
- ondemand_wifi_exclude: HomeNet,OfficeWifi
- ondemand_cellular: true
### Local Installation
- role: local, provider: local
2017-01-04 14:16:55 +00:00
This role is intended to be run for local install onto an Ubuntu server, or onto an unsupported cloud provider's Ubuntu instance. Required variables:
- server - IP address of your server (or "localhost" if deploying to the local machine)
- endpoint - public IP address of the server you're installing on
- ssh_user - name of the SSH user you will use to install on the machine (passwordless login required). If `server=localhost`, this isn't required.
- ca_password - Password for the private CA key
Note that by default, the iptables rules on your existing server will be overwritten. If you don't want to overwrite the iptables rules, you can use the `--skip-tags iptables` flag.
2016-10-13 13:27:06 +00:00
### Digital Ocean
Required variables:
- do_token
- region
Possible options can be gathered calling to https://api.digitalocean.com/v2/regions
2016-10-13 13:27:06 +00:00
### Amazon EC2
Required variables:
- aws_access_key: `AKIA...`
2016-10-13 13:27:06 +00:00
- aws_secret_key
- region: e.g. `us-east-1`
2016-10-13 13:27:06 +00:00
Possible options can be gathered via cli `aws ec2 describe-regions`
2016-10-13 13:27:06 +00:00
Additional variables:
2016-10-13 13:27:06 +00:00
- [encrypted](https://aws.amazon.com/blogs/aws/new-encrypted-ebs-boot-volumes/) - Encrypted EBS boot volume. Boolean (Default: false)
2017-01-10 16:04:29 +00:00
#### Minimum required IAM permissions for deployment:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PreDeployment",
"Effect": "Allow",
"Action": [
"ec2:DescribeImages",
"ec2:DescribeKeyPairs",
"ec2:DescribeRegions",
"ec2:ImportKeyPair",
"ec2:CopyImage"
],
"Resource": [
"*"
]
},
{
"Sid": "DeployCloudFormationStack",
"Effect": "Allow",
"Action": [
"cloudformation:CreateStack",
"cloudformation:UpdateStack",
"cloudformation:DescribeStacks",
"cloudformation:DescribeStackEvents",
"cloudformation:ListStackResources"
],
"Resource": [
"*"
]
},
{
"Sid": "CloudFormationEC2Access",
"Effect": "Allow",
"Action": [
"ec2:CreateInternetGateway",
"ec2:DescribeVpcs",
"ec2:CreateVpc",
"ec2:DescribeInternetGateways",
"ec2:ModifyVpcAttribute",
"ec2:createTags",
"ec2:CreateSubnet",
"ec2:Associate*",
"ec2:CreateRouteTable",
"ec2:AttachInternetGateway",
"ec2:DescribeRouteTables",
"ec2:DescribeSubnets",
"ec2:ModifySubnetAttribute",
"ec2:CreateRoute",
"ec2:CreateSecurityGroup",
"ec2:DescribeSecurityGroups",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RunInstances",
"ec2:DescribeInstances",
"ec2:AllocateAddress",
"ec2:DescribeAddresses"
],
"Resource": [
"*"
]
}
]
}
```
2017-01-01 21:57:08 +00:00
### Google Compute Engine
2016-10-13 13:27:06 +00:00
Required variables:
Refactor to support Ansible 2.8 (#1549) * bump ansible to 2.8.3 * DigitalOcean: move to the latest modules * Add Hetzner Cloud * Scaleway and Lightsail fixes * lint missing roles * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Try to run apt until succeeded * Scaleway modules upgrade * GCP: Refactoring, remove deprecated modules * Doc updates (#1552) * Update README.md Adding links and mentions of Exoscale aka CloudStack and Hetzner Cloud. * Update index.md Add the Hetzner Cloud to the docs index * Remove link to Win 10 IPsec instructions * Delete client-windows.md Unnecessary since the deprecation of IPsec for Win10. * Update deploy-from-ansible.md Added sections and required variables for CloudStack and Hetzner Cloud. * Update deploy-from-ansible.md Added sections for CloudStack and Hetzner, added req variables and examples, mentioned environment variables, and added links to the provider role section. * Update deploy-from-ansible.md Cosmetic changes to links, fix typo. * Update GCE variables * Update deploy-from-script-or-cloud-init-to-localhost.md Fix a finer point, and make variables list more readable. * update azure requirements * Python3 draft * set LANG=c to the p12 password generation task * Update README * Install cloud requirements to the existing venv * FreeBSD fix * env->.env fixes * lightsail_region_facts fix * yaml syntax fix * Update README for Python 3 (#1564) * Update README for Python 3 * Remove tabs and tweak instructions * Remove cosmetic command indentation * Update README.md * Update README for Python 3 (#1565) * DO fix for "found unpermitted parameters: id" * Verify Python version * Remove ubuntu 16.04 from readme * Revert back DigitalOcean module * Update deploy-from-script-or-cloud-init-to-localhost.md * env to .env
2019-09-28 00:10:20 +00:00
- gce_credentials_file: e.g. /configs/gce.json if you use the [GCE docs](https://trailofbits.github.io/algo/cloud-gce.html) - can also be defined in environment as GCE_CREDENTIALS_FILE_PATH
- [region](https://cloud.google.com/compute/docs/regions-zones/): e.g. `useast-1`
### Vultr
Required variables:
- [vultr_config](https://trailofbits.github.io/algo/cloud-vultr.html): /path/to/.vultr.ini
- [region](https://api.vultr.com/v1/regions/list): e.g. `Chicago`, `'New Jersey'`
### Azure
Required variables:
- azure_secret
- azure_tenant
- azure_client_id
- azure_subscription_id
- [region](https://azure.microsoft.com/en-us/global-infrastructure/regions/)
### Lightsail
Required variables:
- aws_access_key: `AKIA...`
- aws_secret_key
- region: e.g. `us-east-1`
Possible options can be gathered via cli `aws lightsail get-regions`
2019-01-24 12:11:34 +00:00
#### Minimum required IAM permissions for deployment:
```
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "LightsailDeployment",
"Effect": "Allow",
"Action": [
2019-02-07 14:09:09 +00:00
"lightsail:GetRegions",
2019-01-24 12:11:34 +00:00
"lightsail:GetInstance",
"lightsail:CreateInstances",
"lightsail:OpenInstancePublicPorts"
],
"Resource": [
"*"
]
}
]
}
```
### Scaleway
Required variables:
- [scaleway_token](https://www.scaleway.com/docs/generate-an-api-token/)
Refactor to support Ansible 2.8 (#1549) * bump ansible to 2.8.3 * DigitalOcean: move to the latest modules * Add Hetzner Cloud * Scaleway and Lightsail fixes * lint missing roles * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Try to run apt until succeeded * Scaleway modules upgrade * GCP: Refactoring, remove deprecated modules * Doc updates (#1552) * Update README.md Adding links and mentions of Exoscale aka CloudStack and Hetzner Cloud. * Update index.md Add the Hetzner Cloud to the docs index * Remove link to Win 10 IPsec instructions * Delete client-windows.md Unnecessary since the deprecation of IPsec for Win10. * Update deploy-from-ansible.md Added sections and required variables for CloudStack and Hetzner Cloud. * Update deploy-from-ansible.md Added sections for CloudStack and Hetzner, added req variables and examples, mentioned environment variables, and added links to the provider role section. * Update deploy-from-ansible.md Cosmetic changes to links, fix typo. * Update GCE variables * Update deploy-from-script-or-cloud-init-to-localhost.md Fix a finer point, and make variables list more readable. * update azure requirements * Python3 draft * set LANG=c to the p12 password generation task * Update README * Install cloud requirements to the existing venv * FreeBSD fix * env->.env fixes * lightsail_region_facts fix * yaml syntax fix * Update README for Python 3 (#1564) * Update README for Python 3 * Remove tabs and tweak instructions * Remove cosmetic command indentation * Update README.md * Update README for Python 3 (#1565) * DO fix for "found unpermitted parameters: id" * Verify Python version * Remove ubuntu 16.04 from readme * Revert back DigitalOcean module * Update deploy-from-script-or-cloud-init-to-localhost.md * env to .env
2019-09-28 00:10:20 +00:00
- region: e.g. `ams1`, `par1`
### OpenStack
You need to source the rc file prior to run Algo. Download it from the OpenStack dashboard->Compute->API Access and source it in the shell (eg: source /tmp/dhc-openrc.sh)
Refactor to support Ansible 2.8 (#1549) * bump ansible to 2.8.3 * DigitalOcean: move to the latest modules * Add Hetzner Cloud * Scaleway and Lightsail fixes * lint missing roles * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Update roles/cloud-hetzner/tasks/main.yml Add api_token Co-Authored-By: phaer <phaer@phaer.org> * Try to run apt until succeeded * Scaleway modules upgrade * GCP: Refactoring, remove deprecated modules * Doc updates (#1552) * Update README.md Adding links and mentions of Exoscale aka CloudStack and Hetzner Cloud. * Update index.md Add the Hetzner Cloud to the docs index * Remove link to Win 10 IPsec instructions * Delete client-windows.md Unnecessary since the deprecation of IPsec for Win10. * Update deploy-from-ansible.md Added sections and required variables for CloudStack and Hetzner Cloud. * Update deploy-from-ansible.md Added sections for CloudStack and Hetzner, added req variables and examples, mentioned environment variables, and added links to the provider role section. * Update deploy-from-ansible.md Cosmetic changes to links, fix typo. * Update GCE variables * Update deploy-from-script-or-cloud-init-to-localhost.md Fix a finer point, and make variables list more readable. * update azure requirements * Python3 draft * set LANG=c to the p12 password generation task * Update README * Install cloud requirements to the existing venv * FreeBSD fix * env->.env fixes * lightsail_region_facts fix * yaml syntax fix * Update README for Python 3 (#1564) * Update README for Python 3 * Remove tabs and tweak instructions * Remove cosmetic command indentation * Update README.md * Update README for Python 3 (#1565) * DO fix for "found unpermitted parameters: id" * Verify Python version * Remove ubuntu 16.04 from readme * Revert back DigitalOcean module * Update deploy-from-script-or-cloud-init-to-localhost.md * env to .env
2019-09-28 00:10:20 +00:00
### CloudStack
Required variables:
- [cs_config](https://trailofbits.github.io/algo/cloud-cloudstack.html): /path/to/.cloudstack.ini
- cs_region: e.g. `exoscale`
- cs_zones: e.g. `ch-gva2`
The first two can also be defined in your environment, using the variables `CLOUDSTACK_CONFIG` and `CLOUDSTACK_REGION`.
### Hetzner
Required variables:
- hcloud_token: Your [API token](https://trailofbits.github.io/algo/cloud-hetzner.html#api-token) - can also be defined in the environment as HCLOUD_TOKEN
- region: e.g. `nbg1`
### Update users
Playbook:
```
users.yml
```
Required variables:
- server - IP or hostname to access the server via SSH
- ca_password - Password to access the CA key
Tags required:
- update-users