This article is a hands-on lab you can follow to learn how to use the OutSystems Private Gateway Feature to connect private resources, not accessible from the internet, to your applications.
The exercise covers:
Creating a Virtual Private Cloud in your AWS account with both public and private subnets.
Installing and configuring a private PostgreSQL instance with sample data.
Installing and configuring the OutSystems Cloud Connector, which establishes a reverse tunnel to your ODC Development stage.
Configuring an ODC Data Fabric connection to the sample PostgreSQL database via the private gateway.
The setup is intended for development purposes and should not be considered a production ready environment. For a production ready environment multiple additional configurations regarding security and availability must be done. Nevertheless the exercise will leave you with an environment you can extend further.
Being an exercise lab the focus is on getting things done and not on explaining concepts and knowledge sharing. I added links in the various sections for further readings.
💡
Note that working through this exercise will incur AWS costs. Even though most of the services and resources used in this exercise are eligible for the free tier, you will be charged for the
NAT Gateway and the associated Elastic IP address.
Prerequisites
To follow this article, you need access to an OutSystems Developer Cloud environment with the licensed Private Gateway feature.
You also need an AWS account with an IAM user that has at least the following permission policies associated:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateVpc",
"ec2:CreateInternetGateway",
"ec2:AttachInternetGateway",
"ec2:CreateSubnet",
"ec2:CreateRouteTable",
"ec2:CreateRoute",
"ec2:AssociateRouteTable",
"ec2:CreateNatGateway",
"ec2:AllocateAddress",
"ec2:RunInstances",
"ec2:DescribeInstances",
"ec2:CreateTags",
"ec2:CreateInstanceConnectEndpoint",
"ec2:DescribeInstanceConnectEndpoints",
"ec2:DescribeVpcs",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeInternetGateways",
"ec2:DescribeRouteTables",
"ec2:DescribeNatGateways",
"ec2:DescribeAddresses"
],
"Resource": "*"
}
]
}
You also need to install and configure aws-cli.
With everything prepared, lets start with setting up a Virtual Private Cloud Resource.
Virtual Private Cloud
💡
The example uses resources in eu-central-1. You can change the region and availability zone to suit your needs.
A Virtual Private Cloud (VPC) is a virtual network in your AWS account. It is a key component of your AWS infrastructure, providing a secure and isolated environment for running AWS resources.
Create a Virtual Private Cloud Resource
To create a new VPC resource, switch to VPC in the AWS console and ensure you have selected the correct region where you want to create the VPC resource.
In the menu, click on Your VPCs and then click on Create VPC.
Select VPC only
Name: odc-vpc.
IPv4 CIDR block: 172.16.0.0/16.
Leave all other settings as default and submit with Create VPC
Additional Documentation
Internet Gateway
OutSystems Cloud Connector needs to make an external connection to our ODC stage to establish a tunnel. Our VPC needs an attached Internet Gateway to allow outbound internet traffic.
Select Internet gateways in the VPC dashboard and click Create internet gateway.
Enter odc-vpc-internet-gateway as the name and click Create internet gateway.
Select the newly created gateway and in the Actions dropdown select Attach to VPC.
Select your created VPC in the list of available VPCs and click on Attach internet gateway.
Additional Documentation
Private Subnet
The private subnet we are creating serves two purposes. First, it will host private resources. Second, it will host our OutSystems Cloud Connector instance to communicate with these private resources.
Select Subnets in the VPC dashboard and click Create subnet.
VPC ID: odc-vpc
Name: odc-vpc-subnet-private
Availability Zone: eu-central-1a
IPv4 subnet CIDR block: 172.16.96.0/20. This creates a subnet with a total of 4096 IP addresses. However, 5 of those addresses are used or reserved by AWS.
Leave the default values for the rest and submit with Create subnet.
Public Subnet
Resources in a private subnet cannot be accessed from the internet and cannot connect to the internet directly. However, the OutSystems Cloud Connector needs to connect to your ODC stage. This can be done using a NAT Gateway associated with a public subnet.
Select Subnets in the VPC dashboard and click Create subnet.
VPC ID: odc-vpc
Name: odc-vpc-subnet-public
Availability Zone: eu-central-1a
IPv4 subnet CIDR block: 172.16.32.0/20
Leave the default values for the rest and click Create subnet.
Elastic IP
Before creating a NAT Gateway in our public subnet, we need to create an Elastic IP address. Elastic IPs are static public internet addresses used by the NAT Gateway to communicate with the internet.
Select Elastic IPs in the VPC dashboard and click Allocate Elastic IP address.
Add a Name tag with the value odc-vpc-elastic-ip.
Leave the default values for the rest and click Allocate.
NAT Gateway
An AWS Network Address Translation (NAT) Gateway is a service that allows resources in a private subnet to connect to the internet while preventing the internet from connecting to private resources.
Select NAT gateways from the VPC menu and click Create NAT gateway
Name: odc-vpc-nat-gateway
Subnet: odc-vpc-subnet-public
Elastic IP allocation ID: odc-vpc-elastic-ip
Leave default values for the rest and submit with Create NAT gateway
Additional Documentation
Route Tables
To complete our initial VPC setup, we need to create two additional route tables, one for each subnet we created.
The route table for the private subnet odc-vpc-subnet-private needs a default route to the NAT Gateway in the public subnet odc-vpc-subnet-public. The public subnet odc-vpc-subnet-public needs a default route to the attached VPC Internet Gateway.
In the Route tables menu, create two new route tables (Create route table). Name the first odc-vpc-rtp-private and the second odc-vpc-rtp-public.
Public Subnet Route Table
In the route tables list, click the route table ID link of odc-vpc-rtp-public.
In the Routes tab, click Edit Routes.
Add a route with destination 0.0.0.0/0, select Internet Gateway from the Target dropdown, and then select the created Internet Gateway.
Submit with Save Changes.
In the Subnet associations tab, click Edit subnet associations.
Select the checkbox next to the subnet odc-vpc-subnet-public and submit with Save associations.
Private Subnet Route Table
In the route tables list, click the route table ID link of odc-vpc-rtp-private.
In the Routes tab, click Edit Routes.
Add a route with destination 0.0.0.0/0, select NAT Gateway from the Target dropdown, and then select the created NAT Gateway.
Submit with Save Changes.
In the Subnet associations tab, click Edit subnet associations.
Select the checkbox next to the subnet odc-vpc-subnet-private and submit with Save associations.
Summary
So far, we have created a Virtual Private Cloud (VPC) with an attached Internet Gateway and two subnets. We have a private subnet that will host internal systems, including the OutSystems Cloud Connector, and a public subnet with a single resource, a NAT Gateway, which allows our private resources to connect to the internet.
Elastic Compute Cloud (EC2)
We are going to launch two EC2 instances. One will host the OutSystems Cloud Connector, and the other will host a PostgreSQL instance.
Launch Instances
Switch to the EC2 Dashboard in AWS Console. In the Instances menu click Launch instances.
Name and tags
Application and OS Images (Amazon Machine Images)
Instance type
- Make sure the t2.micro (Free tier eligible) is selected as Instance type
Key pair (login)
- Select or create a new ssh key pair for the instance
Network Settings
Click on Edit to modify the default settings
Select odc-vpc from the VPC dropdown
Select odc-vpc-subnet-private from the Subnet dropdown
Make sure that Auto-assign public IP is set to Disable
Choose Select existing security group in Firewall (security groups)
Select the default security group in Common security groups
Leave default values for the rest and submit with Launch Instance.
Repeat this and launch a second instance odc-postgresql with the same settings.
Connect Endpoint
Private EC2 instances are not accessible from the internet, so we need to take an extra step to connect with our ec2 instances for installation and configuration.
There are multiple ways to securely access private resources, such as setting up a bastion host that acts as a gateway or setting up a client or site-to-site VPN. In this exercise, we will create and use an EC2 Instance Connect Endpoint to connect to our EC2 instances.
In the VPC dashboard, select the Endpoints menu and click Create endpoint.
Name: odc-vpc-ec2-instance-endpoint
Select EC2 Instance Connect Endpoint in Service Category
Choose odc-vpc from the VPC dropdown
Select the default Security Group
Select odc-vpc-subnet-private from the Subnet dropdown
Leave default values for the rest and submit with Create endpoint
It will take a few minutes for the endpoint to be created. With the settings above, the endpoint allows us to connect to our EC2 instances in the private subnet.
SSH to EC2 Instance
We will use the aws-cli to connect to EC2 instances via SSH through our created EC2 Instance Connect Endpoint. For more connection options, check the documentation.
In the EC2 Dashboard, go to Instances and copy the Instance ID of the EC2 instance you want to connect to.
Open a terminal and enter the following command:
aws ec2-instance-connect ssh --instance-id <your ec2 instance id> --connection-type eice
Upon successful connection you should now see your Amazon Linux prompt.
Summary
In this step, we created two EC2 instances running in the private subnet we set up. One instance will run the OutSystems Cloud Connector, and the other will be used to install a private PostgreSQL instance. We also configured an EC2 Instance Connect Endpoint, which allows us to connect via SSH to both EC2 instances using the AWS CLI.
Install and Configure PostgreSQL
Installation
Connect to the odc-postgresql instance via ssh and execute the following commands
Update System
Run the following command to make sure that your instance has the latest package version installed.
sudo dnf update
Install PostgreSQL
Install PostgreSQL with the following command
sudo dnf install postgresql15.x86_64 postgresql15-server
Initialize Cluster
To initialize a single server cluster run the following command
sudo postgresql-setup --initdb
Start and Enable Service
To start and run PostgreSQL as service execute the following command
sudo systemctl start postgresql
sudo systemctl enable postgresql
systemctl status postgresql
Change postgres system user password
Change the postgres user account password with:
sudo passwd postgres
and set a new password.
Change postgres database user password
To change the postgres database user password run the following commands
sudo su - postgres // su to postgres user
psql -c "ALTER USER postgres WITH PASSWORD '<yourpassword>';" // alter database user password
exit // postgres user logout
Modify PostgreSQL configuration
By default, PostgreSQL listens only to localhost. We will change the configuration file to listen to all IP addresses.
Open the configuration file using the following command
sudo nano /var/lib/pgsql/data/postgresql.conf
Uncomment listen_addresses and replace the value 'localhost' with '*'
Exit nano with CTRL+X and type y to save the changes.
Restart the service with
sudo systemctl restart postgresql
💡
Further installation steps are needed to secure a PostgreSQL instance. However, for our lab, we will keep the installation as it is.
Create a Sample Database
Next we are loading a sample database into our PostgreSQL instance.
Create a database
Connect to your PostgreSQL Instance with
sudo su -u postgres
psql -U postgres
and enter your postgres database user password if prompted.
Then execute the following command in the postgresql shell to create the dvdrental database;
CREATE DATABASE dvdrental;
Check if the database has been created using the \l command. Then type exit to exit psql.
Download sample database
The sample database is provided by PostgreSQLTutorials and can be downloaded and extracted using the following commands.
wget https://www.postgresqltutorial.com/wp-content/uploads/2019/05/dvdrental.zip
unzip dvdrentral.zip
Import sample data
Import the downloaded sample data using the following command
pg_restore -U postgres -d dvdrental ./dvdrental.tar
Verify Import
To verify the imported schema and data start psql with
psql -U postgres
In psql run
The sample database contains 15 tables.
Create Database User
We create a new user with full access to the dvdrental database. We will use this user later to connect OutSystems Data Fabric to this database.
In psql execute the following commands:
CREATE USER odc WITH PASSWORD '<yourpassword>';
GRANT ALL PRIVILEGES ON DATABASE dvdrental TO odc;
Modify Client Authentication
Finally we have to modify the client authentication configuration file to allow our new user to connect to our PostgreSQL Instance.
Open the configuration file with nano using the following command:
sudo nano /var/lib/pgsql/data/pg_hba.conf
Append the following line to the end of the configuration file
host all odc 172.16.96.0/20 md5
Note that the address matches the CIDR of our private subnet, which means the odc user can only connect from this subnet.
Exit nano with CTRL+X and press y to save changes
Restart the PostgreSQL service with the following command
sudo systemctl restart postgresql
Summary
We installed PostgreSQL on our EC2 instance and loaded a sample database. We also created a dedicated user with full privileges to the database.
Activate Private Gateway in ODC Portal
The first step is to activate a private gateway in the ODC portal. You can activate the private gateway feature for each of your available stages separately.
Go to ODC Portal and in the menu, select Configure - Private gateways.
Activate the private gateway for your development stage.
💡
Activation can take several minutes to complete. Do not leave the page until you get an activation complete message.
Immediately copy the displayed Token value to a safe place. It will only be displayed once, and after you leave the page, you will not be able to retrieve it again. However, you can always deactivate and activate the private gateway feature again to generate a new token.
Also, copy the Address value. Both the token and the address are needed for configuring the OutSystems Cloud Connector.
Additional Documentation
Install OutSystems Cloud Connector
Make an SSH connection to the odc-cc EC2 instance.
Download the latest release of OutSystems Cloud Connector from the Cloud Connector GitHub repository. For example to download and extract release 1.3.2 use the following commands:
wget https://github.com/OutSystems/cloud-connector/releases/download/v1.3.2/outsystemscc_1.3.2_linux_amd64.tar.gz
tar xvzf outsystemscc_1.3.2_linux_amd64.tar.gz
unzip ./outsystemscc_1.3.2_linux_amd64.tar.gz
Running Cloud Connector for the first time
To run the cloud connector, you need the address and token values you received when you activated the private gateway feature in the ODC portal. You will also need the IP address of your PostgreSQL EC2 instance, which you can find in the EC2 instance details.
To start the cloud connector, run the following command in the directory where you unzipped the release.
./outsystemscc --header "token: <token value>" <address> R:5432:<postgresql instance private ip>:5432
If the cloud connector successfully connects with your ODC stage, you will see the output Connected (Latency x.xxxxxxms).
To stop cloud connector, press CTRL+C.
Finally we want to configure a service for OutSystems Cloud Connector. We start by creating a dedicated non-root user account to run the service.
Create a user account
sudo adduser odccc
Copy outsystemscc
Next we copy the OutSystems Cloud Connector executable
sudo mkdir -p /opt/odccc
sudo cp outsystemscc /opt/odccc/outsystemscc
sudo chown -R odccc:odccc /opt/odccc
sudo chmod 755 /opt/odccc
sudo chmod 755 /opt/odccc/outsystemscc
Create Service Description File
Open nano with the following command
sudo nano /etc/systemd/system/odccc.service
and add the following content to the file
[Unit]
Description=OutSystems Cloud Connector
After=network.target
[Service]
ExecStart=/opt/odccc/outsystemscc --header "token:<token value>" <address> R:5432:<instance ip address>:5432
User=odccc
Group=odccc
Restart=on-failure
[Install]
WantedBy=multi-user.target
Exit nano with CTRL+X and press y to save the file
Enable and Start the Service
To enable and start the OutSystems Cloud Connector Service run the following commands
sudo systemctl daemon-reload
sudo systemctl enable odccc
sudo systemctl start odccc
Check the status of your service with sudo systemctl status odccc and verify the Connected state.
Summary
In this part, we activated the Private Gateway feature in the ODC Portal and installed and configured the OutSystems Cloud Connector to run as a service on our odc-cc EC2 instance.
💡
Take note that we hard-coded all the values in our configurations, which is not the best practice. A better approach is to use AWS Systems Manager Parameter Store to store the configuration options for the OutSystems Cloud Connector. In your EC2 instance, you can set these values using the User Data configuration option as environment variables and then use those in the ExecStart of the service.
Data Fabric Connection
After completing the installation and configuration, we are now ready to add a connection to our ODC stage.
The configuration process is straightforward. We begin by adding a new connection.
Add Connection
Open ODC Portal and click on Integrate - Connections and the on Create connection
From the Service Provider list select PostgreSQL and click Confirm
Connection Name: ec2_postgres
Make sure the Development tab is selected
Username: odc (the one we created when we configured PostgreSQL)
Password: The password you set for the odc database user
Host: secure-gateway (This tells ODC to use the connected cloud connector)
Port: 5432
Database: dvdrental
SSL Mode: Disabled (we haven't configured any security on PostgreSQL so we have to disable it)
Click on Test Connection to check if ODC can connect to the dvdrental database through the running cloud connector. After that, click Save to create the new connection.
Select Entities
Next, we need to select (and optionally configure) the entities we want to use in our ODC applications.
In the Connections list, click "Select entities" in the ec2_postgres table row.
In the detail screen, check the boxes next to customer, store, and address.
Optionally, you can configure additional settings, such as the internal names of entities and attributes.
Submit your selection by clicking Save.
Using entities in Applications
Just like any other public entity, Data Fabric entities are public elements with the source matching the connection name you defined.
Summary
In this section, we added a Data Fabric connection to the ODC Portal and selected external entities we want to make available within our ODC applications.
The End?
You have reached the end of this exercise lab, and I hope you successfully connected to the sample database through the OutSystems Cloud Connector. If any parts need more details or are unclear, please let me know.
As mentioned earlier, this is a basic environment for development and testing purposes only. For a more mature production environment, you should consult an AWS expert to set up a secure and highly available cloud connector installation. Speaking of high availability, check out this article by Paul Ho on Improving Availability of ODC Cloud Connector.
Follow me on LinkedIn to receive notifications whenever I publish something new.
