Introduction
Update Sep 2024: Change default agent to thin-edge. Using go-c8y-cli as default local proxy for Passthrough
Update Jan 2024: I’ve updated some links and added a chapter to tunnel a local HTTP proxy directly to the C8Y UI.
Last year, I published an article roughly describing the Cloud Remote Access Feature of Cumulocity IoT.
Since then I received a lot of requests on how to use it in detail and decided to write another knowledge base article going into detail and covering all of the requests I received. So the following questions should be addressed in this article:
- What is the Cloud Remote Access Feature?
- Is VPN required for that?
- Which agents support it / How can an agent support cloud remote access?
- How can I have a Web SSH / VNC Connection my Device?
- How can I make use of the Passthrough configuration?
- What are the benefits of using the Passthrough configuration instead of Web SSH / VNC?
- How can I connect to a device using Passthrough configuration and the local proxy?
- Is there a simple way to bridge a local HTTP Server to make it accessible via Cumulocity IoT?
Let’s get started with an overview about the Cloud Remote Access Feature!
Cloud Remote Access
The Cloud Remote Access Feature of Cumulocity IoT allows you to simply connect to your devices using SSH, Telnet, VNC or any other TCP-based protocol. The good thing about that is you don’t need any VPN tunnel by getting the same grade of security a VPN tunnel would provide. This is achieved by having a microservice running in Cumulocity IoT tunneling all protocols through a secure web socket connection and taking care of the authentication.
When using the SSH, Telnet or VNC configuration of the Cloud Remote Access, the tunneled connection is terminated within the Cloud Remote Access Microservice, and the output is visualized in a web terminal in a browser using xTerm.js
Here is an example how this looks like in the browser:
Sometimes it is not sufficient that we are limited to a web terminal and that the Cloud Remote Access Microservice terminates the connection. Therefore we have introduced the Passthrough Configuration.
As you can see from the sketch above in Passthrough configurations, no clients of the microservice are used. Instead it forwards the packets to another web socket channel where a local proxy is running. The local proxy connects to the web socket provided by the microservice, provides TCP sockets where native clients can be connected and of course tunnels the packets receiving from the clients through the web socket and vice versa.
The logic on the device side is always the same.
So let’s sum this up:
- If you want to have a web terminal using SSH, telnet or VNC, you can achieve that with the according provided configuration within the Cumulocity IoT platform
- If this is not sufficient and you want to use native clients, you can make use of the Passthrough configuration in combination of a local proxy.
- In both scenarios, you just need an agent implementing the Cloud Remote Access feature
Cloud Remote Access enabled agents
There are some agents available which are already implementing the Cloud Remote Access feature on device side.
Cloud Remote Access Agent
If you want to try out the Cloud Remote Access Feature and nothing else, the Cloud Remote Access Agent is the one you should go for. The good thing is it also contains a python module, which can by easily integrated into your python agent, enabling the cloud remote access feature only in a few steps. Have a look here for more details! The python package is also available at pypi
thin-edge.io
Also the thin-edge.io implements the cloud remote access feature fully as one of many Device Management Features Cumulocity IoT is providing. So if you are looking for a production grade full working agent you should go for the thin-edge.io
https://thin-edge.github.io/thin-edge.io/operate/c8y/remote_access_with_cumulocity/
Device Management Reference Agent
–DEPRECATED–
The device management reference agent has been explained in detail in one of my last articles: Getting Started with Cumulocity IoT Device Management
The cloud remote access module from the agent is part of this agent as it leverages docker containers to quickly create new devices and it comes with a preconfigured SSH & VNC Server. It is one of the fastest options to try out Cloud Remote Access as it provides a docker image on docker hub and a package on pypi.
Cumulocity Linux Agent
The Linux Agent is a lightweight agent implemented in C for linux operated devices. It also has implemented the Cloud Remote Access Feature.
c8yMQTT Raspberry Pi Sense Hat Agent
This agent is very similar to the Device Management Reference Agent. It is implemented in Python and supports the Cloud Remote Access Feature. It is mainly developed for the raspberry pi in combination with a sense hat.
https://github.com/Cumulocity-IoT/c8yMQTT
Steps to establish a Web SSH/VNC connection
As thin-edge.io has everything built-in, I will use it as a reference in this guide.
I will use the demo container at is has everything that we need. Same should valid using a real device like the raspberry pi.
Using the other agent requires additional steps like setting up a SSH and VNC Server on the Device.
NOTE: This is only used for demonstration purposes. Of course you can use any agent of the list above or custom agent to follow the steps below but need to set up additional components manually.
Prerequisites
- Docker must be installed
- Git or any other git client of your choice
- A Cumulocity (Trial) tenant with Cloud Remote Access Feature enabled.
- go-c8y-cli as a local proxy and REST API helper tool
NOTE: The Cloud Remote Access Feature is an additional feature and must be assigned to your Tenant. Please ask your contact or administrator to add it to your tenant
1. Start thin-edge
This is only needed if you don’t have any agent running already. Please go on with step 2 if you already have an agent running supporting Cloud Remote Access.
To start the thin-edge demo container you can follow the Getting started guide in the repo or perform the following steps:
- Download the docker compose file from the repository
curl -LSs https://raw.githubusercontent.com/thin-edge/tedge-demo-container/main/demos/docker-compose/device/docker-compose.yaml > docker-compose.yaml
Or alternatively you can download it using wget
wget https://raw.githubusercontent.com/thin-edge/tedge-demo-container/main/demos/docker-compose/device/docker-compose.yaml
- Start the docker-compose project (in the background)
docker compose up -d
- Bootstrap the device
docker compose exec tedge bootstrap.sh
After the successful bootstrap your thin-edge device and the child devices should be added to your tenant.
2. Assign Cloud Remote Access roles to users
To use the Cloud Remote Access functionality, we have to add the existing roles to the users who should be allowed to use this feature. As this feature is pretty powerful (and sometimes destructive), it should be only assigned to experts who exactly know what they are doing.
Go to Administration → Roles → Add global role (on top right).
Give it the name “Cloud Remote Access Role” and make sure to check the “Admin” permission for “Remote Access”.
Next go to Users → select your user and assign the just created role to it. Don’t forget to save your user.
NOTE: If you don’t see the “Remote Access” Tab in Device Management most probably it is because you haven’t assigned the role to your user
3. Configure Web SSH Remote Connection
Now that we have the required permission to use the Cloud Remote Access, we can switch to Device Management Application. Go to your device. You should see now the “Remote Access” Tab. Click on Add Endpoint.
The following details must be provided:
- Name of endpoint: Local SSH Server (in my case)
- Protocol: SSH
- Sign-in method: Username and password
- Host: 127.0.0.1 (the SSH Server is running in the same host where the device agent is running)
- Port: 22
- Username: iotadmin (preconfigured in thin-edge demo container, must be changed of course in different environments)
- **Password:**iotadmin (preconfigured in thin-edge demo container, must be changed of course in different environments
The host key is optional and will be accepted and stored on first connect.
Please click on Save to store the configuration.
4. Start Web SSH Remote Connection
As we have configured our first SSH connection, we can now test it by clicking on “Connect” Symbol on the right in the table.
A pop-up will be opened with the web ssh client session. After some seconds, you should see the terminal of your device.
Now you can enter any command of your choice e.g. “df -h” to check available disk space.
NOTE: If you don’t see the terminal but are getting a time-out or other error message please check if the device is online and available. Also check the provided credentials and connection details you’ve provided in step 3.
5. Configure and start the VNC remote connection
Hint This is not working out of the box as the thin-edge demo container does not come with an UI nor VNC server. Use the dm-agent or use any other hardware like the raspberry pi if you want to try it out.
Let’s repeat step 3 and 4 for a VNC remote connection.
Click an “Add Endpoint.”
The following details must be provided:
- Name: Local VNC
- Protocol: VNC
- Sign-in method: Password only
- Host: 127.0.0.1
- Port: 5901 (notice the change to 5901 here)
- Password: test456# (preconfigured in dm-agent docker file, must be changed of course in different environments)
Click on “Save.”
Finally click on “Connect” to start the VNC Remote connection. You should see the desktop screen now in a browser window.
NOTE: If you don’t see the desktop and are getting an error, try different ports e.g. 5902. Sometimes the vnc server increases the port automatically especially when the agent is restarted multiple times.
Steps to setup a passthrough connection
As described above, the passthrough is the most advanced way to use cloud remote access. You might wonder why there is no “Passthrough” option in the drop-down box when creating a new Remote Access Endpoint. This is a systemwide option that is disabled by default on all our public instances (cumulocity.com / eu-latest.cumulocity.com and so on). On dedicated instances, this option can be set to enabled, and you will see an option in the drop-down box when creating a new endpoint. Still you can try the passthrough option on public instances. Follow the steps below to do so.
NOTE: This guide only helps to demonstrate the passthrough capabilities. If you want to use this in production and on a mass scale, please get in contact with your responsible Software AG contact or Software AG support to make sure not running to any issues.
1. Create a passthrough endpoint
First of all, we need to create a passthrough endpoint. The easiest way would be using the UI. So if you can see the passthrough option in the UI, use that to create an endpoint. You just need to provide a name, a URL and a port. No credentials are needed as they are provided by the client.
If you don’t see the passthrough option in the selection box, you can create an endpoint using the REST API or c8y-go-cli.
Let’s start with the c8y-go-cli. Assuming you already have created and selected a session with set-session:
c8y remoteaccess configurations create-passthrough --device tedge_168da5b44d5a --name Passthrough --port 22
That’s it.
As an alternative you can use Postman or a similar REST Client to execute the following call:
POST <host>/service/remoteaccess/devices/<InternalDeviceID>/configurations
Headers:
Basic Auth: <base64 encoded (yourC8YUser : yourPassword)>
Content-Type: application/json
Accept: application/json
Payload:
{
"name": "Passthrough",
"hostname": "<hostname of local server>",
"port": <port of local server>,
"protocol": "PASSTHROUGH",
"credentialsType": "NONE"
}
Example to connect to the local SSH Server:
POST https://<<yourTenantURL>>/service/remoteaccess/devices/97282922/configurations
Headers:
<see above>
Payload:
{
"name": "Passthrough",
"hostname": "127.0.0.1",
"port": 22,
"protocol": "PASSTHROUGH",
"credentialsType": "NONE"
}
As response we get back:
{
"attrs": {},
"id": "2",
"name": "Passthrough",
"hostname": "127.0.0.1",
"port": 22,
"protocol": "PASSTHROUGH",
"credentials": {
"type": "NONE"
}
}
Also in the UI, when we refresh the Cloud Remote Access view on our Device, we should see the endpoint now:
2. Install c8y-go-cli
As we’ve seen in the introduction section for passthrough, we need an agent supporting remote access and a local proxy running on the client side or on a server, we can connect to with our local clients. For that purpose we use a the go-c8y-cli which includes local proxy capabilities:
As an alternative we can also use a dedicated tool:
If you haven’t installed c8y-go-cli please follow the guide here to install it.
Afterwards follow the Getting Started guide to setup a session.
3. Connecting to device
Now we can run the local proxy using the c8y-go-cli.
The local proxy supports multiple modes to connect. For us, two modes are of interest:
- Proxy Server Mode - which opens a local server listening on a port a a local client can connect
- connect ssh Mode - directly use local installed ssh client to establish a ssh session
3.1 Using connect ssh mode
Next we can already establish a ssh connection. Make sure that you’ve open ssh locally installed on your computer by entering ssh in your terminal.
Now we can run the connect mode of the local proxy:
c8y remoteaccess connect ssh --device <external ID of device> --user iotadmin
It will prompt you for the password for the user “iotadmin” and directly connects you to the device.
3.2 Using server mode
As an alternative, we can use the server mode, so we don’t have the requirement to have ssh installed but can use any other client like putty to connect to our local proxy instance. Also you can tunnel any other TCP port with it like local HTTP-Server that are proxied to the client.
Using c8y-go-cli local proxy you can listen on the following mediums:
- Unix socket
- TCP Port
- Standard Input/Output (stdio)
With the command below using the 0 we will use a local free and random port.
c8y remoteaccess server --device tedge_168da5b44d5a --listen "127.0.0.1:0"
This command opens a random port (42103 in my example) which we can use to connect with a client of our choice:
In my example I use putty:
Connection is established successfully:
4. Further use cases for using Passthrough
There are much more use cases for using the Passthrough endpoint but not only SSH.
For example you can bridge a local accessible Web UI using the Passthrough configuration. To do so, you just need to configure the internal URL and port in the Passthrough config. If you now connect the local proxy in server mode you can enter localhost and the generated port in your browser to access the local User Interface.
Basically any TCP-Port can be bridged that way, not only SSH, VNC or HTTP, see next chapter for an example bridging HTTP Server to the cloud.
Setup a cloud http proxy
Using the Passthrough configuration you can also easily tunnel a local HTTP server which is only accessible for the device. The following open-source repo provides a microservice, who takes care about the connection handling and a UI Plugin that let you visualize any User Interface of that local HTTP Server directly on the device in Device Management.
See this example where a local node-red server is tunneled and visualized on a C8Y device directly from the cloud.
Summary
In this guide, I demonstrated almost all possibilities you have when using the Cloud Remote Access Feature including SSH, VNC and Passthrough endpoints on a step-by-step approach and using the DM reference agent. It should help to understand the basic concepts and demonstrate on an example of how to perform the steps using Cumulocity IoT.
If you have any questions or feedback feel free to approach me! In case of issues with used components like dm-agent or local proxy, feel free to open an issue directly in the Github projects.
This article is part of the TECHniques newsletter blog - technical tips and tricks for the Software AG community. Subscribe to receive our quarterly updates or read the latest issue.