How to fix redash saml's self-sign Certificate Authority?

13/09/2023 @ Saigon Redash

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 345, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py", line 462, in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host=MY_ADFS_SERVER.LOCAL', port=443):
Max retries exceeded with url: /FederationMetadata/2007-06/FederationMetadata.xml
(Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

Why does it happen?

Redash trying to get SAML’s FederationMetadata.xml from a self-sign ADFS server. Python package named certifi did not update with your new CA’s certificate.

How to solve?

Find certifi’s cacert.pem and update it. In terminal, type python to access its interactive shell. Then, type the following command.

>>> import certifi
>>> certifi.where()
'/usr/local/lib/python3.7/site-packages/certifi/cacert.pem'

In this case, it’s /usr/local/lib/python3.7/site-packages/certifi/cacert.pem.

Append your CA’s certificate to cacert.pem.

$ cat my-ca.crt >> /usr/local/lib/python3.7/site-packages/certifi/cacert.pem

How to test?

In terminal, type python to access its interactive shell. Then, type the following command.

>>> import requests
>>> requests.request("GET", "https://YOUR_ADFS_DOMAIN/FederationMetadata/2007-06/FederationMetadata.xml")

Good luck!

Reference

Sheng JH (13 September 2021) How to fix “certificate verify failed: self signed certificate in certificate chain”or “certificate verify failed: unable to get local issuer certificate” error while verifying certificates to SSL enabled website in python3 scripts, IBM website, accessed on 13 September 2023. https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/sheng-jie-han/2021/06/03/how-to-fix-certificate-verify-failed-self-signed-c

How to add Certificate Authority (CA) in Fedora to support chain certificate?

09/09/2023 @ Saigon SSL

Step 1. Using chrome to extract certificates.

[1] Open certificate viewer in Google Chrome

Only need to use extract CA’s certificate. Export it with file extension named .pem

Please take a note that, update-ca-trust determines certificate format using file header which locates in very first bytes in the binary file. Eventhough you save certificates with .crt , .cer, it’s still .pem.

To determine file format, you shoule use command file, for example $ file file_name.

To illustrate this point. I’ll give an example.

####### List all file, take a look at the file extension, .crt and .pem
$ ls -l
'Default Trust_DigiCert Global Root CA.crt'
'Default Trust_DigiCert Global Root CA.pem'


####### Determine file format with command named `file`
$ file *
Default Trust_DigiCert Global Root CA.crt: PEM certificate
Default Trust_DigiCert Global Root CA.pem: PEM certificate

Step 2. Copy certificate authority’s certificate to `/etc/pki/ca-trust/source/anchors`

Step 3. Update `/etc/ssl/certs/ca-certificates.crt`

$ sudo update-ca-trust extract

You can check this file /etc/ssl/certs/ca-certificates.crt to ensure that it is updated.

Step 4. Testing

This is an image before update-ca-trust

And, this is an image after update-ca-trust.

Good luck!

GlobalProtect 6.0.7

07/09/2023 @ Saigon etc

File Name	Platform	md5sum	URL
GlobalProtect64-6.0.7.msi	Window	92ea9d9b994c8ab11c236e0b740b628e	dropbox link
PanGPLinux-6.0.7.tgz	Linux	378514202fbc893c397e1aec87b06c58	dropbox link
GlobalProtect-6.0.7.pkg	Mac	c6d3b506e291bbe1cb5a87488f1209ab	dropbox link

How to compile xmrig on Fedora?

07/11/2023 @ Saigon Mining Rig

This is a repost from https://xmrig.com/docs/miner/build/fedora which then I can save my time searching in future.

I. Basic build

Basic build is good for local machine, because it is easy, but if you need to run the miner on other machines please take a look at advanced build.

$ sudo dnf install -y git make cmake gcc gcc-c++ libstdc++-static libuv-static hwloc-devel openssl-devel
$ git clone https://github.com/xmrig/xmrig.git
$ mkdir xmrig/build && cd xmrig/build
$ cmake ..
$ make -j$(nproc)

II. Advanced build

We use build_deps.sh script to build recent versions of libuv, openssl and hwloc as static libraries.

$ sudo dnf install -y git make cmake gcc gcc-c++ libstdc++-static automake libtool autoconf perl
$ git clone https://github.com/xmrig/xmrig.git
$ mkdir xmrig/build
$ cd xmrig/scripts && ./build_deps.sh && cd ../build
$ cmake .. -DXMRIG_DEPS=scripts/deps
$ make -j$(nproc)

Use command ldd xmrig to verify binary dependencies.

III. Reference

XMRig build fedora, https://xmrig.com/docs/miner/build/fedora

Sơ đồ kích thước fan máy tính 120mm

01/09/2023 @ Saigon etc

Reference:

GDSTIME 5Volt 120mm Fan, 1500RPM 120mm x 25mm DC Brushless Cooling Fan, Amazon 2019, https://www.amazon.com/GDSTIME-5Volt-1500RPM-Brushless-Cooling/dp/B07Y7XB6G2

How to change DBeaver's heap size?

23/08/2023 @ Saigon etc

Edit /usr/share/dbeaver-ce/dbeaver.ini, change the -Xms and -Xmx. In the example belows, check line 20 and 21.

-Xms: It is used for setting the initial and minimum heap size. I set to 1GB.
-Xmx: It is used for setting the maximum heap size. I set to 6GB.

On Window, it’s C:\Users\user\AppData\Local\DBeaver\dbeaver.ini or C:\Program Files\DBeaver\dbeaver.ini

On Mac, it’s /Applications/DBeaver.app/Contents/Eclipse/dbeaver.ini

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
-vmargs
-XX:+IgnoreUnrecognizedVMOptions
-Dosgi.requiredJavaVersion=17
--add-modules=ALL-SYSTEM
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.nio.charset=ALL-UNNAMED
--add-opens=java.base/java.text=ALL-UNNAMED
--add-opens=java.base/java.time=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/jdk.internal.vm=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--add-opens=java.base/sun.security.ssl=ALL-UNNAMED
--add-opens=java.base/sun.security.util=ALL-UNNAMED
-Xms1G
-Xmx6G
-Ddbeaver.distribution.type=rpm

Ender-3 Pro, Cura settings

16/08/2023 @ Saigon 3D Print

I. Low Quality - 0.25mm

Latest Update: 16/8/2023

Downdown cura profile here: Dropbox link

JamDB Oracle - How to fix :badmatch error when fetch big file?

11/08/2023 @ Saigon Elixir

Example error:

iex(2)> App.OracleRepo.get(App.BigBigBigDocument, 3573564)
** (DBConnection.ConnectionError) {:badmatch, <<253, 249, 243, 207, 63, 219, 35, 182, 23, 177, 72, ...>>}
    (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:932: Ecto.Adapters.SQL.raise_sql_call_error/1
    (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:847: Ecto.Adapters.SQL.execute/6
    (ecto 3.8.4) lib/ecto/repo/queryable.ex:221: Ecto.Repo.Queryable.execute/4
    (ecto 3.8.4) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
    (ecto 3.8.4) lib/ecto/repo/queryable.ex:147: Ecto.Repo.Queryable.one/3
iex(2)> {"level":"error","message":"Jamdb.Oracle (#PID<0.889.0>) disconnected: ** (DBConnection.ConnectionError) {:badmatch, <<253, 249, 243,  173, 123, 177, 72, ...>>}","timestamp":"2023-08-11T03:51:46.563Z"}

Solution: Add connection parameter named :read_timeout in repo config. In the follow example, I change :read_timeout to 5 minutes, see line 11. Besides, I add :timeout to :infinity, see line 10.

1
2
3
4
5
6
7
8
9
10
11
config :app, App.OracleRepo,
  hostname: "******",
  port: 1521,
  database: "******",
  username: "******",
  password: "******",
  stacktrace: true,
  show_sensitive_data_on_connection_error: true,
  pool_size: 10,
  timeout: :infinity,
  parameters: [ read_timeout: :timer.minutes(5)]

How to configure KeyCloak to support Redash SAML SSO?

10/10/2023 @ Saigon Redash

I. KeyCloak Configuration

Step 1: Create a new realm

First of all, let start with definition of reaml, it’s what I got from KeyCloak official website.

Realms A realm manages a set of users, credentials, roles, and groups. A user belongs to and logs into a realm. Realms are isolated from one another and can only manage and authenticate the users that they control.

Realms (Tiếng Việt): cõi, địa hạt, vương quốc, vùng Một realm quản lý một tập hợp user, thông tin xác thực - credential, role và group. Một user sẽ thuộc một realm, một realm sẽ có nhiều user, user sẽ đăng nhập vào reaml mà nó thuộc về. Một server KeyCloak tạo được rất nhiều realm, và chúng bị cô lập với nhau. Những realm này chỉ có thể quản lý và cho đăng nhập những user mà chúng quản lý.

As the definition of realm , this step is only used for testing only, normaly, if your KeyCloak did create a realm, you can skip this step and go to step 2.

Step 2: Create new reaml’s client & configure it

In new client form, I would like to input the following parameters, then submit.

Client ID: redash
Client Protocol: saml
Client SAML Endpoint: ignored this field.

After that, edit that new client named redash

Client ID: redash

Name: Redash
Description: empty
Enabled: on
Always Display in Console: off
Consent Required: off
Login Theme: keycloak
Client Protocol: saml
Include AuthnStatement: on
Include OneTimeUse Condition: off
Force Artifact Binding: off
Sign Documents: off
Sign Assertions: on
Signature Algorithm: RSA_SHA256
SAML Signature Key Name: KEY_ID
Canonicalization Method: EXCLUSIVE_WITH_COMMENTS
Encrypt Assertions: off
Client Signature Required: off
Force POST Binding: off
Front Channel Logout: off
Force Name ID Format: off
Allow ECP Flow: off
Name ID Format: email
Root URL: ignored
Valid Redirect URIs: https://redash.local-domain.com/*
Base URL: https://redash.local-domain.com/
Master SAML Processing URL: https://redash.local-domain.com/saml/callback?org_slug=default
IDP Initiated SSO URL Name: ignored
Logo URL: ignored
Policy URL: ignored
Terms of service URL: ignored
IDP Initiated SSO Relay State: ignored

Step 3: Configure client’s mappers

In the saml response that Redash expected to received, it requires

First Name (original), this attibute name is FirstName
Last Name (original), this attribute name is LastName

However, in the KeyCloak, the attribute names are different from what Redash expected, as a consequence, we need to configure client’s mappers For first name and last name, use Add Builtin feature.

[6] First name and last name mapper detail

X500 Surname
- Property: lastName
- Friendly Name: LastName
- SAML Attribute Name: LastName
X500 GivenName
- Property: firstName
- Friendly Name: FirstName
- SAML Attribute Name: FirstName

II. Redash SAML Configuration

After login using admin credential, go to Settings → General → Saml

SAML Enabled: Enable (Dynamic)
SAML Metadata URL: https://keyclock.local-domain.com/apps/keycloak/realms/{REALM_NAME}/protocol/saml/descriptor
SAML Entity ID: redash
SAML NameID Format: urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress

You are done! Enjoy!

III Reference

[saml] Signature missing for assertion, rockxsj, 2018, https://github.com/getredash/redash/issues/2977

Talend, How to export log in JSON Format?

21/06/2023 @ Saigon Talend

Step 1: Go to File → Project Properties

Step 2: Go to log4j section and Activate log4j in components (Log4j version: log4j2)

In the Log4j template, use the following template.

<?xml version="1.0" encoding="UTF-8"?>
<Configuration >
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout  >
        <pattern>
          { "timestamp":"%d{ISO8601}{GMT+0}", "level":"%level", "category":"%c", "message":"%enc{%m}{JSON}" }%n
        </pattern>
      </PatternLayout>
    </Console>
  </Appenders>

  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="Console" />
    </Root>
  </Loggers>
</Configuration>

Step 3: Create tJava in Talend Job to export log

Basic settings code:

String message = "Hello World";
log.fatal(message);
log.error(message);
log.warn(message);
log.info(message);
log.debug(message);
log.trace(message);

Advanced settings code:

import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;

You then can RUN your talend job, the output should look like the following.

References

Configure PatternLayout, jmcollin92, 2021, May 24, https://stackoverflow.com/questions/29387007/does-log4j-support-json-format#comment119606534_37455869.
Handle special character in json log message, Nishan B, 2022, November 25, https://stackoverflow.com/questions/29387007/does-log4j-support-json-format#comment131629876_37455869.

Previous Next

Hexa's Blog

Hexa's Blog

How to fix redash saml's self-sign Certificate Authority?

Why does it happen?

How to solve?

How to test?

Reference

How to add Certificate Authority (CA) in Fedora to support chain certificate?

Step 1. Using chrome to extract certificates.

Step 2. Copy certificate authority’s certificate to `/etc/pki/ca-trust/source/anchors`

Step 3. Update `/etc/ssl/certs/ca-certificates.crt`

Step 4. Testing

GlobalProtect 6.0.7

How to compile xmrig on Fedora?

I. Basic build

II. Advanced build

III. Reference

Sơ đồ kích thước fan máy tính 120mm

How to change DBeaver's heap size?

Ender-3 Pro, Cura settings

I. Low Quality - 0.25mm

JamDB Oracle - How to fix :badmatch error when fetch big file?

How to configure KeyCloak to support Redash SAML SSO?

I. KeyCloak Configuration

Step 1: Create a new realm

Step 2: Create new reaml’s client & configure it

Step 3: Configure client’s mappers

II. Redash SAML Configuration

III Reference

Talend, How to export log in JSON Format?

Step 1: Go to File → Project Properties

Step 2: Go to log4j section and Activate log4j in components (Log4j version: log4j2)

Step 3: Create tJava in Talend Job to export log

References

Hexa's Blog

Hexa's Blog

Why does it happen?

How to solve?

How to test?

Reference

Step 1. Using chrome to extract certificates.

Step 2. Copy certificate authority’s certificate to /etc/pki/ca-trust/source/anchors

Step 3. Update /etc/ssl/certs/ca-certificates.crt

Step 4. Testing

I. Basic build

II. Advanced build

III. Reference

I. Low Quality - 0.25mm

I. KeyCloak Configuration

Step 1: Create a new realm

Step 2: Create new reaml’s client & configure it

Step 3: Configure client’s mappers

II. Redash SAML Configuration

III Reference

Step 1: Go to File → Project Properties

Step 2: Go to log4j section and Activate log4j in components (Log4j version: log4j2)

Step 3: Create tJava in Talend Job to export log

References

Step 2. Copy certificate authority’s certificate to `/etc/pki/ca-trust/source/anchors`

Step 3. Update `/etc/ssl/certs/ca-certificates.crt`