398 lines
12 KiB
Markdown
398 lines
12 KiB
Markdown
# {...} UpClapi
|
|
|
|
<img style="margin-top: 1em;width: 500px;border: 0" alt="Fork me on GitHub" src="logo/upclapi.svg?sanitize=true">
|
|
|
|
## Declarative & command-line interface for [UpCloud](https://upcloud.com) resources
|
|
|
|
This was created since December 2020 to:
|
|
|
|
- to use YAML config definition files for resources and services
|
|
- create,delete,start,stop [UpCloud servers](https://upcloud.com/products/cloud-servers/).
|
|
- modify, delete [UpCloud storages](https://upcloud.com/products/maxiops-storage/).
|
|
- Run commands with **ssh** in [UpCloud servers](https://upcloud.com/products/cloud-servers/).
|
|
- create servers [Ansible inventories](https://docs.ansible.com/ansible/latest/index.html)
|
|
|
|
> TskSrvcs can be <u>declared</u> and <u>grouped together</u> in [YAML](https://en.wikipedia.org/wiki/YAML) files, some of them can use target resource UUID
|
|
|
|
All in a [GO](https://golang.org/) binary program thanks to [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api)
|
|
|
|
It clearly does not try to replace tools like [Terraform](https://www.terraform.io/) or [Ansible](https://docs.ansible.com/),etc to manage **Infraestructure** rather as an alternative for some cases, like:
|
|
|
|
- simple servers provisioning
|
|
- servers tsksrvcs: start, stop, delete
|
|
- modify network interfaces
|
|
- float ip management
|
|
- modify storage
|
|
- automatic resources scale
|
|
- run commands on servers via **ssh** (output is captured)
|
|
- create [Ansible inventories](https://docs.ansible.com/ansible/latest/index.html)
|
|
- resources lists in **json**
|
|
- prototyping
|
|
- resources costs
|
|
...
|
|
- As a provider tool for other **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools: [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala)
|
|
|
|
Since March 2021 [UpCloud](https://github.com/UpCloudLtd) made available a [UpCloud CLI - upctl](https://github.com/UpCloudLtd/upcloud-cli#upcloud-cli---upctl) to allow resources control from command line.
|
|
[{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) was create to be used with a [Declarative](https://en.wikipedia.org/wiki/Declarative_programming) approach to manage infrastructures although some tasks can be solved with imperative command calls, resources relationship and their dependencies should be set and declared somewhere at least for monitoring,management,etc.
|
|
|
|
Both developments ([upctl](https://github.com/UpCloudLtd/upcloud-cli#upcloud-cli---upctl) and [{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi)) use [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api) extensively.
|
|
|
|
> [{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) is used as a tool to provision and manage **UpCloud** ressources for **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) with [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) as major tools.
|
|
## Requirements for build
|
|
|
|
- Go `1.16+`
|
|
- Go Modules defined in [go.mod](https://rlung.librecloud.online/LibreCloud/Upclapi/src/branch/master/go.mod)
|
|
|
|
## Why is named like {...}UpClapi
|
|
|
|
This development is fully focus on **UpCloud** as resources provider and use extensively [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api).
|
|
So this is what it use and his focus, but *{...}* define what it is, a group of resources than can be created, set and manage by custom definition or declaration with a name you chose.
|
|
|
|
Short version, can be interpreted as:
|
|
> whatever group **{...}** of **UpCloud** resources managed via UpCloud **API**
|
|
|
|
## Download Release binary
|
|
|
|
1 - Copy <u>attachment url</u> from [releases page](https://rlung.librecloud.online/LibreCloud/Upclapi/releases)
|
|
|
|
2 - Use this **shell** command changing <u>attachment URL</u>:
|
|
|
|
```bash
|
|
curl -sLS "** attachment URL **" | gunzip > upclapi ; chmod +x upclapi
|
|
````
|
|
|
|
3 - Test it by running in **shell**:
|
|
|
|
```bash
|
|
./upclapi -h
|
|
```
|
|
|
|
4 - Move this file to something like: /usr/local/bin or wherever you like.
|
|
|
|
## Build from source
|
|
|
|
- Clone this repository and go to folder:
|
|
|
|
```bash
|
|
git clone https://rlung.librecloud.online/LibreCloud/Upclapi.git
|
|
cd Upclapi
|
|
```
|
|
|
|
- Build:
|
|
|
|
```bash
|
|
make
|
|
```
|
|
|
|
- Install ($GOPATH/bin):
|
|
|
|
```bash
|
|
make install
|
|
```
|
|
|
|
### Run examples
|
|
|
|
- [UpCloud](https://upcloud.com) Credentilas
|
|
Use evironment variables like this:
|
|
|
|
```bash
|
|
export UPCLOUD_USERNAME="api-username"
|
|
export UPCLOUD_PASSWORD="api-password"
|
|
```
|
|
|
|
As an alternative this can be set as in an enviroment file with paths:
|
|
/etc/upcloud/.env or /usr/local/etc/upcloud/.env
|
|
|
|
Another alternative is to use default [UpCloud CLI - upctl](https://github.com/UpCloudLtd/upcloud-cli#quick-start) config set path:
|
|
$HOME/.config/upctl.yaml
|
|
|
|
If creadentials are not found, this message will be displayed:
|
|
|
|
```txt
|
|
Auth info: Username must be specified
|
|
or
|
|
Auth info: Password must be specified
|
|
```
|
|
|
|
As a more secure option it is possible to encrypt sensitive informations like:
|
|
|
|
- Credentials (username/password)
|
|
- Data config file in yaml
|
|
|
|
- Version
|
|
|
|
```bash
|
|
# Getting version
|
|
upclapi version
|
|
# or upclapi -v
|
|
```
|
|
|
|
- Help
|
|
|
|
```bash
|
|
# Getting help
|
|
upclapi --help
|
|
# or upclapi -h
|
|
```
|
|
|
|
```txt
|
|
Usage of upclapi:
|
|
-c string
|
|
command to run [
|
|
createserver, infoserver, restartserver, startserver, stopserver, modifyserver, deleteserver,
|
|
infofloatip, movefloatip, modifyip,
|
|
infotags, addtags, deletetags (use -t keep to keep storages)
|
|
liststorages, fixstorage (-t size | part0 | part1 | final), deletestorage, modifystorage, createstorageimage (-t title),
|
|
inventory, pubhosts, prvhosts
|
|
runssh, runcmd
|
|
] (default "infoserver")
|
|
-cmd string
|
|
run [ssh] command
|
|
-f string
|
|
path to data file (default "datacfg.yaml")
|
|
-id string
|
|
resource name or uuid
|
|
-kdr string
|
|
use coder
|
|
-o string
|
|
output format (attached for infofloatip)
|
|
-sid string
|
|
ssh id for ssh commands (or defaul id_rsa)
|
|
-t string
|
|
target item for command
|
|
-v version
|
|
```
|
|
|
|
- Create Server
|
|
|
|
```bash
|
|
./build/upclapi -c createserver -f [yaml-config-file]
|
|
```
|
|
|
|
- Info Server
|
|
|
|
```bash
|
|
./build/upclapi -c infoserver -f [yaml-config-file]
|
|
```
|
|
|
|
```bash
|
|
./build/upclapi -c infoserver -id server-hostname
|
|
```
|
|
|
|
- Move a Float IP
|
|
|
|
```bash
|
|
./build/upclapi -c movefloatip -f [yaml-config-file]
|
|
```
|
|
|
|
- Get attached IP to Float IP
|
|
|
|
```bash
|
|
./build/upclapi -c infofloatip -id float-ip ( or -f [yaml-config-file] ) -o attached
|
|
```
|
|
|
|
```bash
|
|
./build/upclapi -c movefloatip -id float-ip-address::target-hostname
|
|
```
|
|
|
|
- Run SSH on Server
|
|
(yaml-config-file or -id flag)
|
|
<small>if no **-sid** flag *id_pub* in HOME will be used by default</small>
|
|
|
|
```bash
|
|
./build/upclapi -c runssh -f [yaml-config-file] -cmd remote-command-to-execute [-sid ssh/key_filename]
|
|
./build/upclapi -c runssh -id hostname -cmd remote-command-to-execute [-sid ssh/key_filename]
|
|
```
|
|
|
|
- Create Servers [Ansible Inventory](https://docs.ansible.com/ansible/latest/index.html)
|
|
(yaml-config-file has to include **cluster** config section)
|
|
|
|
```bash
|
|
./build/upclapi -c inventory -f [yaml-config-file]
|
|
```
|
|
|
|
> User account persmissions may not allow some tsksrvcs to create or deleted
|
|
> To fully manage, UpCloud owner account is mandatory
|
|
|
|
- To show info about coverage (will open in browser):
|
|
|
|
```bash
|
|
make show_coverage
|
|
```
|
|
|
|
- That's all! 🎉
|
|
|
|
## YAML resources declaration
|
|
|
|
```yaml
|
|
# Controller to manage and monitor
|
|
cntrllrs:
|
|
- cldPath: relative-path-for-controller
|
|
host: hostname
|
|
masterPath: ''
|
|
sshaccess:
|
|
host: hostname
|
|
keyPath: ''
|
|
password: ''
|
|
port: 22
|
|
user: ssh-user
|
|
utype: key
|
|
domainName: domain-name
|
|
floatIP: ''
|
|
group: group-name
|
|
group_path: relative-group-path
|
|
hostPrefix: hostprefix-
|
|
mainName: main-name
|
|
# UpCloud as provider
|
|
provider: upcloud
|
|
resolvDNS: ''
|
|
servers:
|
|
# Repeat this section for each host
|
|
- apps: # Apps to install
|
|
- critical: no
|
|
liveness: ''
|
|
name: none
|
|
path: ''
|
|
req: ''
|
|
target: ''
|
|
arch: linux_amd64
|
|
backup:
|
|
- ''
|
|
clients:
|
|
- ''
|
|
cluster:
|
|
role: role-tag
|
|
hostname: hostname
|
|
loginUser:
|
|
sshKeys:
|
|
- ''
|
|
metadata: 1
|
|
networks:
|
|
- access: utility
|
|
family: IPv4
|
|
source_ip_filtering: yes
|
|
- access: public
|
|
family: IPv4
|
|
source_ip_filtering: yes
|
|
- access: private # If use SDN
|
|
family: IPv4
|
|
ipaddress: 'ip-for-sdn'
|
|
network: "sdn-uuid"
|
|
source_ip_filtering: no
|
|
plan: "1xCPU-1GB" # UpCloud server plan
|
|
rules:
|
|
- ''
|
|
sshAccess:
|
|
host: hostname
|
|
keyPath: ''
|
|
password: ''
|
|
port: 22
|
|
user: "ssh-user"
|
|
utype: key
|
|
storages:
|
|
- action: clone
|
|
size: 20
|
|
finalSize: 20 # These 3 lines can be used to create partitions
|
|
makefs: ""
|
|
partSizes: ''
|
|
source: source-name
|
|
storage: UUID-for-source-image
|
|
tier: maxiops
|
|
title: Title-for-this-storage
|
|
tags:
|
|
- cd
|
|
- ci
|
|
timeZone: UTC
|
|
title: titls-of-this-host
|
|
tsksrvcs: # Tasks & Services to install,maintain,monitor, etc.
|
|
- critical: no
|
|
liveness: ''
|
|
name: sysconfig
|
|
path: ''
|
|
req: ''
|
|
target: ''
|
|
useFloatIP: false
|
|
userData: ''
|
|
zone: es-mad1 # UpCloud Zone
|
|
|
|
```
|
|
|
|
Attributes and values are declared in [main.go](https://rlung.librecloud.online/LibreCloud/Upclapi/src/branch/master/main.go) and *yaml parsed*.
|
|
|
|
A lot of definitions (__apps, tsksrvcs, cntrllrs, etc.__) are used mainly for other **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools: [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala). These [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools developed in [Rust](https://www.rust-lang.org/) declare attributes and values with [Dhall language](https://dhall-lang.org/) and then converted to formats like yaml and json (so types are consistent and grouped accordly).
|
|
|
|
[{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) consumes <u>ONLY yaml</u> config/declaration files, and can be used for standalone proposal with yaml resources declaration files (in such case set empty definitions for: __apps, tsksrvcs, cntrllrs__)
|
|
|
|
## Environment
|
|
|
|
Main vars:
|
|
|
|
[UpCloud API](https://developers.upcloud.com) connetions
|
|
|
|
- UPCLOUD_USERNAME with username
|
|
- UPCLOUD_PASSWORD with password
|
|
|
|
- KUPCLAPI includes user + password encrypted
|
|
|
|
Use to preset settings:
|
|
|
|
| Arg | Env. var |
|
|
|---|---|
|
|
|-c|UPCLAPI_COMMAND|
|
|
|-f| UPCLAPI_DATACFG|
|
|
|-id|UPCLAPI_ID|
|
|
|-cmd|UPCLAPI_RUNCMD|
|
|
|-kdr|UPCLAPI_ENCODER|
|
|
|-o|UPCLAPI_OUT|
|
|
|
|
## Coders and encoders
|
|
|
|
It is possible to encrypt sensitive informations like:
|
|
|
|
- Credentials (username/password)
|
|
- Data config file in yaml
|
|
|
|
Coders are called using operating system command (exec)
|
|
|
|
> [Tecoder]() from [ZTerton]()
|
|
> backend developments is fully supported
|
|
|
|
Coder definition to be used:
|
|
| var | content |
|
|
|---|---|
|
|
| cmd | command-path |
|
|
| abbrv | abbreviation |
|
|
| decoder | decoder-args |
|
|
| fdecoder | file-decoder-args |
|
|
| encoder | encoder-args |
|
|
| fencoder | file-encoder-args |
|
|
|
|
## Author
|
|
|
|
- [Jesús Pérez](https://info.jesusperez.pro).
|
|
|
|
## Article assistance
|
|
|
|
If you want to say «thank you»:
|
|
|
|
1. Twit about article [on your Twitter](https://twitter.com/intent/tweet?text=Let%27s%20write%20config%20for%20your%20Golang%20web%20app%20on%20right%20way%20%E2%80%94%20YAML%20%F0%9F%91%8C%20https%3A%2F%2Fdev.to%2Fkoddr%2Flet-s-write-config-for-your-golang-web-app-on-right-way-yaml-5ggp).
|
|
2. Add a GitHub Star and make Fork to this repository.
|
|
3. Donate some money to project author via PayPal: [@paypal.me/](https://paypal.me/).
|
|
4. Join UpCloud at my [referral link](https://upcloud.com/signup/?promo=CVR337) (your profit is **\$25** credits and I will receive \$50 worth of free credits).
|
|
|
|
Thanks for your support! 😘
|
|
|
|
## References
|
|
|
|
[UpCloudLtd go API](https://github.com/UpCloudLtd/upcloud-go-api)
|
|
|
|
[Let's write config for your Golang web app on right way — YAML 👌](https://dev.to/koddr/let-s-write-config-for-your-golang-web-app-on-right-way-yaml-5ggp). Published @ 13 Feb 2020.
|
|
|
|
[Go language: crypto / SSH executes remote commands](https://developpaper.com/go-language-crypto-ssh-executes-remote-commands/)
|
|
|
|
[golang-enter-ssh-sudo-password-on-prompt-or-exit](https://gist.github.com/boyzhujian/73b5ecd37efd6f8dd38f56e7588f1b58)
|
|
|
|
## License
|
|
|
|
MIT
|