add ngrok developemnt and curl request helpers (#392)

.gitignore

@@ -4,3 +4,7 @@ build/
 
 cmd/mdmctl/mdmdctl
 cmd/micromdm/micromdm
+
+tools/ngrok/config_root
+tools/ngrok/filerepo
+tools/ngrok/env

tools/api/README.md

@@ -0,0 +1,54 @@
+A sample of the common API requests are available as `curl` scripts. To use them, install all the requirements and follow the steps as described.
+
+## Requirements
+
+- [jq](https://stedolan.github.io/jq/)
+  `brew install jq`
+
+## Setup
+Create an `env` file and define environment variables you will need to talk to the server. 
+This env file will be sourced by the scripts.
+
+Contents of `env` file:
+```
+# the value of the -api-key flag that MicroMDM was started with.
+export API_TOKEN=supersecret
+export SERVER_URL=https://mdm.acme.co
+```
+
+In your shell, set the environment variable `MICROMDM_ENV_PATH` to point to your env file.
+Do this every time you open a new shell to work with the scripts in this folder.
+```
+export MICROMDM_ENV_PATH="$(pwd)/env"
+```
+
+## Usage examples
+
+```
+# get all devices from micromdm. returns JSON.
+./tools/api/get_devices
+
+# use jq to filter response. For example, to get the udid of the first device.
+./tools/api/get_devices | jq .devices[0].udid -r
+
+# send a push notification to a device UDID
+./tools/api/send_push_notification <device-udid>
+
+# combine sending a push notification with the get devices request.
+$udid=(tools/api/get_devices |jq .devices[0].udid -r)
+./tools/api/send_push_notification $udid
+```
+
+The `api` folder groups all MDM commands in the `commands` folder. Use these for scheduling commands against a remote device. All of these take the device udid as a first argument and might take additional arguments when necessary.
+
+```
+# install a configuration profile
+./tools/api/commands/install_profile $udid /path/to/profile.mobileconfig
+
+# remove a configuration profile
+./tools/api/commands/remove_profile $udid Your-Profile-PayloadIdentifier
+```
+
+All the scripts in this folder are tiny. Consider reading them to get an understanding of the API.
+Also consider learning `jq`. 
+MicroMDM returns and accepts JSON for all of the API endpoints, and `jq` is a powerful tool for reading and manipulating JSON data.

tools/api/commands/device_lock

@@ -0,0 +1,12 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+endpoint="v1/commands"
+jq -n \
+  --arg request_type "DeviceLock" \
+  --arg udid "$1" \
+  --arg pin $2 \
+  '.udid = $udid 
+  |.pin = $pin 
+  |.request_type = $request_type
+  '|\
+  curl -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint" -d@-

tools/api/commands/install_profile

@@ -0,0 +1,12 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+endpoint="v1/commands"
+jq -n \
+  --arg request_type "InstallProfile" \
+  --arg udid "$1" \
+  --arg payload $(cat $2|base64) \
+  '.udid = $udid
+  |.payload = $payload 
+  |.request_type = $request_type
+  '|\
+  curl -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint" -d@-

tools/api/commands/remove_profile

@@ -0,0 +1,12 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+endpoint="v1/commands"
+jq -n \
+  --arg request_type "RemoveProfile" \
+  --arg udid "$1" \
+  --arg identifier $2 \
+  '.udid = $udid
+  |.identifier = $identifier
+  |.request_type = $request_type
+  '|\
+  curl -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint" -d@-

tools/api/commands/restart_device

@@ -0,0 +1,10 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+endpoint="v1/commands"
+jq -n \
+  --arg request_type "RestartDevice" \
+  --arg udid "$1" \
+  '.udid = $udid 
+  |.request_type = $request_type
+  '|\
+  curl -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint" -d@-

tools/api/get_devices

@@ -0,0 +1,4 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+endpoint="v1/devices"
+curl -s -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint" 

tools/api/send_push_notification

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+source $MICROMDM_ENV_PATH
+endpoint="push"
+curl -u "micromdm:$API_TOKEN" "$SERVER_URL/$endpoint/$1"

tools/ngrok/README.md

@@ -0,0 +1,108 @@
+# Local Development with ngrok
+
+When developing and testing MicroMDM you will likely want to enroll devices. Normally that means either generating or getting new TLS certificates and hosting micromdm on some publicly accessible server.
+`ngrok` is a tool that makes it easy to expose the MicroMDM server as a public URL that your devices can enroll to. 
+
+The scripts in this will help you get MicroMDM running for local development. 
+First, install the listed requirements. 
+
+Next, make sure you've cloned the `micromdm` repo exactly as described in the [contributing guide](../../CONTRIBUTING.md). Follow the steps in the same document to downloads required dependencies and build `micromdm` and `mdmctl`.
+
+Finally, follow the described workflow for setting up your development server.
+
+## Requirements
+
+- [go](https://golang.org/)
+  `brew install go`
+
+- [jq](https://stedolan.github.io/jq/)
+  `brew install jq`
+
+- [ngrok](https://ngrok.com)
+  `brew cask install ngrok`
+
+## Workflow
+
+Follow the steps in order, and use separate terminal windows for running ngrok and micromdm. These are long running processes, and you will want to see the output.
+Some of the steps are only required for first time setup.
+
+_Note:_ When the ngrok tunnel expires, you have to re-enroll the test device. The `configure_mdmctl` updates the DEP profile, so all you need to do is either reset the VM, or run `sudo /usr/bin/profiles -e -N` if testing with a real Mac.
+
+1. Run ngrok. A tunnel will be opened. If you're using the free version of `ngrok` this URL is only valid for 8 hours, but you can always start with a new URL.
+```
+ngrok http 8080
+```
+Make sure to open a browser tab at `http://localhost:4040`. The ngrok web interface shows all the requests/responses between `micromdm` and its clients and can be invaluable for debugging.
+
+2. Create a env file and define environment variables you will need for micromdm. This is a first time setup only step.
+
+Contents of `env` file:
+```
+export API_TOKEN=supersecret
+export PUSH_CERTIFICATE_PRIVATE_KEY_PASSWORD=supersecret
+export SERVER_URL="$(curl -s localhost:4040/api/tunnels | jq '.tunnels[]|.public_url' -r |grep https)"
+```
+
+3. Set the `MICROMDM_ENV_PATH` variable to be the path of your `env` file. This will be used to source the file in all the helper scripts. Do this once in every new terminal window you open while developing with micromdm.
+
+```
+export MICROMDM_ENV_PATH="$(pwd)/env"
+```
+
+4. Start `micromdm`. Do this every time you rebuild the binary and when your `ngrok` tunnel expires.
+
+```
+./tools/ngrok/start_server
+```
+
+The server will run on port `8080`. The default configuration is minimal, but you can pass any additional flags as arguments to the script. For example `./tools/ngrok/start_server -http-debug`.
+
+5. Configure the Push certificate and DEP tokens. This is a first time setup only step.
+If you already have the `MDM CSR` option in your enterprise portal, follow [these steps](https://github.com/micromdm/micromdm/wiki/Generating-MicroMDM-MDM-Certificates#generating-micromdm-mdm-certificates). If you don't, you can get a free MDM certificate from [mdmcert.download](https://github.com/micromdm/micromdm/wiki/mdmcert.download) or export the device management certificate from profile manager.
+
+Configure `mdmdctl`:
+
+```
+source $MICROMDM_ENV_PATH
+./build/darwin/mdmctl config set -name ngrok -api-token=$API_TOKEN -server-url=$SERVER_URL
+./build/darwin/mdmctl config switch -name=ngrok
+```
+
+Upload push certificate:
+```
+./build/darwin/mdmctl mdmcert upload  \
+  -cert "$path_to/mdm-certificates/push_certificate.pem" \
+  -private-key "$path_to/mdm-certificates/PushCertificatePrivateKey.key" \
+  -password $PUSH_CERTIFICATE_PRIVATE_KEY_PASSWORD
+```
+
+Get a DEP OAuth token at deploy.apple.com
+```
+./build/darwin/mdmctl get dep-tokens -export-public-key /path/to/save/public.pem
+
+# after you upload the public key to your DEP portal, you'll get back a .p7m token file.
+./build/darwin/mdmctl apply dep-tokens -import "$dev_path/dep/local_dep_smime.p7m"
+```
+
+6. Configure mdmctl and start using micromdm. Do this every time your ngrok tunnel expires.
+```
+# substitute A_SERIAL_NUMBER for your test device. This serial number will be used to apply a DEP profile.
+./tools/ngrok/configure_mdmctl A_SERIAL_NUMBER
+```
+
+`micromdm` and `mdmctl` should now be configured and you can begin testing.
+
+## Everyday workflow.
+
+```
+# In your first terminal window. Don't forget to open http://localhost:4040
+ngrok http 8080
+
+# In a second terminal window
+export MICROMDM_ENV_PATH="$(pwd)/env"
+./tools/ngrok/start_server
+
+# In a third terminal window.
+export MICROMDM_ENV_PATH="$(pwd)/env"
+./tools/ngrok/configure_mdmctl
+```

tools/ngrok/configure_mdmctl

@@ -0,0 +1,21 @@
+#!/bin/bash
+source $MICROMDM_ENV_PATH
+micromdm_go_path="$(go env GOPATH)/src/github.com/micromdm/micromdm"
+mdmctl="$micromdm_go_path/build/$(uname |tr [:upper:] [:lower:])/mdmctl"
+config_root="$micromdm_go_path/tools/ngrok/config_root"
+
+$mdmctl config set -name ngrok -api-token=$API_TOKEN -server-url=$SERVER_URL
+$mdmctl config switch -name=ngrok
+
+# DEP profile configuration. Comment out if not using DEP.
+$mdmctl apply dep-profiles -template | jq \
+  --arg server_url "$SERVER_URL/mdm/enroll" \
+  --arg org_magic "$(uuidgen)" \
+  --arg serial_number "$1" \
+  --arg profile_name "$(whoami)-ngrok" \
+  '.profile_name |= $profile_name
+  |.url |= $server_url
+  |.devices |= [$serial_number]
+  |.org_magic |= $org_magic 
+  |.await_device_configured |= true' > $config_root/dep_profile.json
+$mdmctl apply dep-profiles  -f $config_root/dep_profile.json

tools/ngrok/start_server

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+source $MICROMDM_ENV_PATH
+micromdm_go_path="$(go env GOPATH)/src/github.com/micromdm/micromdm"
+micromdm="$micromdm_go_path/build/$(uname |tr [:upper:] [:lower:])/micromdm"
+config_root="$micromdm_go_path/tools/ngrok/config_root"
+mkdir -p $config_root
+
+$micromdm serve \
+  -api-key $API_TOKEN \
+  -server-url=$SERVER_URL \
+  -config-path $config_root \
+  -tls=false "$@"