Upclapi/README.md

# {...} UpClapi

<img style="margin-top: 1em;width: 500px;border: 0" alt="Fork me on GitHub" src="logo/upclapi.svg?sanitize=true">

## Declarative & command-line interface for [UpCloud](https://upcloud.com) resources

This was created since December 2020 to:

- to use YAML config definition files for resources and services
- create,delete,start,stop [UpCloud servers](https://upcloud.com/products/cloud-servers/).
- modify, delete [UpCloud storages](https://upcloud.com/products/maxiops-storage/).
- Run commands with **ssh** in [UpCloud servers](https://upcloud.com/products/cloud-servers/). 
- create servers [Ansible inventories](https://docs.ansible.com/ansible/latest/index.html)

 > TskSrvcs can be <u>declared</u> and <u>grouped together</u> in [YAML](https://en.wikipedia.org/wiki/YAML) files, some of them can use target resource UUID 

 All in a [GO](https://golang.org/) binary program thanks to [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api)

It clearly does not try to replace tools like [Terraform](https://www.terraform.io/) or [Ansible](https://docs.ansible.com/),etc to manage **Infraestructure** rather as an alternative for some cases, like:

- simple servers provisioning
- servers tsksrvcs: start, stop, delete
- modify network interfaces
- float ip management
- modify storage
- automatic resources scale
- run commands on servers via **ssh** (output is captured)
- create [Ansible inventories](https://docs.ansible.com/ansible/latest/index.html)
- resources lists in **json**
- prototyping
- resources costs
...
- As a provider tool for other **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools: [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala)

Since March 2021 [UpCloud](https://github.com/UpCloudLtd) made available a [UpCloud CLI - upctl](https://github.com/UpCloudLtd/upcloud-cli#upcloud-cli---upctl) to allow resources control from command line.
[{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) was create to be used with a [Declarative](https://en.wikipedia.org/wiki/Declarative_programming) approach to manage infrastructures although some tasks can be solved with imperative command calls, resources relationship and their dependencies should be set and declared somewhere at least for monitoring,management,etc.

Both developments ([upctl](https://github.com/UpCloudLtd/upcloud-cli#upcloud-cli---upctl) and [{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi)) use [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api) extensively.

> [{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) is used as a tool to provision and manage **UpCloud** ressources for **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) with [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) as major tools.
## Requirements for build

- Go `1.16+`
- Go Modules defined in [go.mod](https://rlung.librecloud.online/LibreCloud/Upclapi/src/branch/master/go.mod)

## Why is named like {...}UpClapi

This development is fully focus on **UpCloud** as resources provider and use extensively [Official UpCloudLtd GO API](https://github.com/UpCloudLtd/upcloud-go-api).
So this is what it use and his focus, but *{...}* define what it is, a group of resources than can be created, set and manage by custom definition or declaration with a name you chose.

Short version, can be interpreted as:
> whatever group **{...}** of **UpCloud** resources managed via UpCloud **API**

## Download Release binary

1 - Copy <u>attachment url</u> from [releases page](https://rlung.librecloud.online/LibreCloud/Upclapi/releases)

2 - Use this **shell** command changing <u>attachment URL</u>:

```bash
curl -sLS "** attachment URL **" | gunzip > upclapi  ; chmod +x upclapi 
````

3 - Test it by running in **shell**:

```bash
./upclapi -h
```

4 - Move this file to something like: /usr/local/bin or wherever you like.

## Build from source

- Clone this repository and go to folder:

```bash
git clone https://rlung.librecloud.online/LibreCloud/Upclapi.git
cd Upclapi
```

- Build:

```bash
make
```

- Install ($GOPATH/bin):

```bash
make install
```

### Run examples

- [UpCloud](https://upcloud.com) Credentilas
Use evironment variables like this:

```bash
export UPCLOUD_USERNAME="api-username"
export UPCLOUD_PASSWORD="api-password"
```

As an alternative this can be set as in an enviroment file with paths:
 /etc/upcloud/.env or /usr/local/etc/upcloud/.env

Another alternative is to use default [UpCloud CLI - upctl](https://github.com/UpCloudLtd/upcloud-cli#quick-start) config set path:
$HOME/.config/upctl.yaml

If creadentials are not found, this message will be displayed:

```txt
Auth info: Username must be specified
or
Auth info: Password must be specified
```

As a more secure option it is possible to encrypt sensitive informations like:

- Credentials (username/password)
- Data config file in yaml

- Version

```bash
# Getting version 
upclapi version
# or upclapi -v
```

- Help

```bash
# Getting help 
upclapi --help  
# or upclapi -h 
```

```txt
Usage of upclapi:
  -c string
    command to run [
    createserver, infoserver, restartserver, startserver, stopserver, modifyserver, deleteserver,
    infofloatip, movefloatip, modifyip,
    infotags, addtags, deletetags (use -t keep to keep storages)
    liststorages, fixstorage (-t size | part0 | part1 | final), deletestorage, modifystorage, createstorageimage (-t title),
    inventory, pubhosts, prvhosts
    runssh, runcmd
    ] (default "infoserver")
  -cmd string
    run [ssh] command
  -f string
    path to data file (default "datacfg.yaml")
  -id string
    resource name or uuid
  -kdr string
    use coder
  -o string
    output format (attached for infofloatip)
  -sid string
   ssh id for ssh commands (or defaul id_rsa)
  -t string
    target item for command
  -v  version
```

- Create Server

```bash
./build/upclapi  -c createserver -f [yaml-config-file]
```

- Info Server

```bash
./build/upclapi  -c infoserver -f [yaml-config-file] 
```

```bash
./build/upclapi  -c infoserver -id server-hostname
```

- Move a Float IP

```bash
./build/upclapi  -c movefloatip -f [yaml-config-file]
```

- Get attached IP to Float IP

```bash
./build/upclapi  -c infofloatip -id float-ip ( or -f [yaml-config-file] ) -o attached
```

```bash
./build/upclapi  -c movefloatip -id float-ip-address::target-hostname
```

- Run SSH on Server
  (yaml-config-file or -id flag)
  <small>if no **-sid** flag *id_pub* in HOME will be used by default</small>

```bash
./build/upclapi  -c runssh -f [yaml-config-file] -cmd remote-command-to-execute [-sid ssh/key_filename]
./build/upclapi  -c runssh -id hostname -cmd remote-command-to-execute [-sid ssh/key_filename]
```

- Create Servers [Ansible Inventory](https://docs.ansible.com/ansible/latest/index.html)
  (yaml-config-file has to include **cluster** config section)

```bash
./build/upclapi  -c inventory -f [yaml-config-file] 
```

> User account persmissions may not allow some tsksrvcs to create or deleted 
> To fully manage, UpCloud owner account is mandatory

- To show info about coverage (will open in browser):

```bash
make show_coverage
```

- That's all! 🎉

## YAML resources declaration

```yaml
# Controller to manage and monitor
cntrllrs:
  - cldPath: relative-path-for-controller
    host: hostname
    masterPath: ''
    sshaccess:
      host: hostname
      keyPath: ''
      password: ''
      port: 22
      user: ssh-user
      utype: key
domainName: domain-name
floatIP: ''
group: group-name
group_path: relative-group-path
hostPrefix: hostprefix-
mainName: main-name
# UpCloud as provider
provider: upcloud
resolvDNS: ''
servers:
# Repeat this section for each host 
  - apps: # Apps to install
      - critical: no
        liveness: ''
        name: none
        path: ''
        req: ''
        target: ''
    arch: linux_amd64
    backup:
      - ''
    clients:
      - ''
    cluster:
      role: role-tag
    hostname: hostname
    loginUser:
      sshKeys:
        - ''
    metadata: 1
    networks:
      - access: utility
        family: IPv4
        source_ip_filtering: yes
      - access: public
        family: IPv4
        source_ip_filtering: yes
      - access: private # If use SDN
        family: IPv4
        ipaddress: 'ip-for-sdn'
        network: "sdn-uuid"
        source_ip_filtering: no
    plan: "1xCPU-1GB" # UpCloud server plan
    rules:
      - ''
    sshAccess:
      host: hostname
      keyPath: ''
      password: ''
      port: 22
      user: "ssh-user"
      utype: key
    storages:
      - action: clone
        size: 20
        finalSize: 20 # These 3 lines can be used to create partitions 
        makefs: ""
        partSizes: ''
        source: source-name
        storage: UUID-for-source-image
        tier: maxiops
        title: Title-for-this-storage
    tags:
      - cd
      - ci
    timeZone: UTC
    title: titls-of-this-host
    tsksrvcs: # Tasks & Services to install,maintain,monitor, etc.
      - critical: no
        liveness: ''
        name: sysconfig
        path: ''
        req: ''
        target: ''
    useFloatIP: false
    userData: '' 
    zone: es-mad1 # UpCloud Zone

```

Attributes and values are declared in [main.go](https://rlung.librecloud.online/LibreCloud/Upclapi/src/branch/master/main.go) and *yaml parsed*.

A lot of definitions (__apps, tsksrvcs, cntrllrs, etc.__) are used mainly for other **LibreClouds** [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools: [CloudMandala](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala) and [Zteron](https://rlung.librecloud.online/LibreCloud/CloudMandala#cloudmandala). These [Klouds](https://rlung.librecloud.online/LibreCloud/Klouds#klouds) tools developed in [Rust](https://www.rust-lang.org/) declare attributes and values with [Dhall language](https://dhall-lang.org/) and then converted to formats like yaml and json (so types are consistent and grouped accordly).

[{...}UpClapi](https://rlung.librecloud.online/LibreCloud/Upclapi#upclapi) consumes <u>ONLY yaml</u> config/declaration files, and can be used for standalone proposal with yaml resources declaration files (in such case set empty definitions for: __apps, tsksrvcs, cntrllrs__)

## Environment

Main vars:

[UpCloud API](https://developers.upcloud.com) connetions

- UPCLOUD_USERNAME with username
- UPCLOUD_PASSWORD with password

- KUPCLAPI includes user + password encrypted

Use to preset settings:

| Arg | Env. var |
|---|---|
|-c|UPCLAPI_COMMAND|
|-f| UPCLAPI_DATACFG|
|-id|UPCLAPI_ID|
|-cmd|UPCLAPI_RUNCMD|
|-kdr|UPCLAPI_ENCODER|
|-o|UPCLAPI_OUT|

## Coders and encoders

It is possible to encrypt sensitive informations like:

- Credentials (username/password)
- Data config file in yaml

Coders are called using operating system command (exec)

> [Tecoder]() from [ZTerton]()
> backend developments is fully supported

Coder definition to be used:
| var | content |
|---|---|
| cmd | command-path |
| abbrv | abbreviation |
| decoder | decoder-args |
| fdecoder | file-decoder-args |
| encoder | encoder-args |
| fencoder | file-encoder-args |

## Author

- [Jesús Pérez](https://info.jesusperez.pro).

## Article assistance

If you want to say «thank you»:

1. Twit about article [on your Twitter](https://twitter.com/intent/tweet?text=Let%27s%20write%20config%20for%20your%20Golang%20web%20app%20on%20right%20way%20%E2%80%94%20YAML%20%F0%9F%91%8C%20https%3A%2F%2Fdev.to%2Fkoddr%2Flet-s-write-config-for-your-golang-web-app-on-right-way-yaml-5ggp).
2. Add a GitHub Star and make Fork to this repository.
3. Donate some money to project author via PayPal: [@paypal.me/](https://paypal.me/).
4. Join UpCloud at my [referral link](https://upcloud.com/signup/?promo=CVR337) (your profit is **\$25** credits and I will receive \$50 worth of free credits).

Thanks for your support! 😘

## References

[UpCloudLtd go API](https://github.com/UpCloudLtd/upcloud-go-api)

[Let's write config for your Golang web app on right way — YAML 👌](https://dev.to/koddr/let-s-write-config-for-your-golang-web-app-on-right-way-yaml-5ggp). Published @ 13 Feb 2020.

[Go language: crypto / SSH executes remote commands](https://developpaper.com/go-language-crypto-ssh-executes-remote-commands/)

[golang-enter-ssh-sudo-password-on-prompt-or-exit](https://gist.github.com/boyzhujian/73b5ecd37efd6f8dd38f56e7588f1b58)

## License

MIT