NEX-195: Allow the frontend to access the backend via internal url in runtime

[vermario]

This PR introduces the concept of an internal url to access drupal from the frontend.

The backend needs to be accessible by a public URL for these reasons:

1. the editors need to log into the site and do their content editing work
2. in our example, there is a flow where the user asks to register on the frontend, gets an email with the link to go to drupal and set the password, gets redirected back
3. the build of the site is done in our case by CircleCI and it's not feasible to allow access to the internal url because the circle ci agent does not have a predictable IP.

But, it would be useful for the node.js process running in next.js to access the backend also with an internal URL. In SILTA, this means that the data flows inside the same cluster, and does not involve our CDN.

One thing that we had to take into consideration was the fact that we are using next-image, and both the internal and external urls need to be allowed as source for the images.

I have added the same concept to the lando and ddev setups, because hopefully this would allow us to catch subtle bugs in case we are using the wrong internal/external url for drupal there.

Summary of the changes:

Configuration Updates:

• .circleci/config.yml: Added environment variables DRUPAL_BASE_URL_INTERNAL and DRUPAL_BASE_URL_INTERNAL_IMAGES to handle internal URLs and ensure they are used within CircleCI. [1] [2] [3]
• .ddev/config.yaml: Added DRUPAL_BASE_URL_INTERNAL and DRUPAL_BASE_URL_INTERNAL_IMAGES for local development.
• .lando.yml: Added DRUPAL_BASE_URL_INTERNAL and DRUPAL_BASE_URL_INTERNAL_IMAGES for Lando environments.
• silta/silta-next.yml: Added DRUPAL_BASE_URL_INTERNAL and DRUPAL_BASE_URL_INTERNAL_IMAGES for Silta environments.

Code Updates:

• next/next.config.mjs: Updated the image hostname handling to use both external and internal Drupal URLs, ensuring valid URLs are provided. [1] [2]
• next/src/app/api/health/route.ts: Changed the health check endpoint to use the internal Drupal URL.
• next/src/auth.ts: Updated OAuth token fetching to use the internal Drupal URL. [1] [2]
• next/src/components/formatted-text.tsx: Modified image source URLs to use the internal Drupal URL.
• next/src/env.ts: Added internal Drupal URL environment variables to the environment configuration. [1] [2]
• next/src/lib/drupal/drupal-client.ts: Updated the GraphQL client to use the internal Drupal URL.

[hkirsman]

I liked also this explanation from Slack thread:

Main issue:

If the frontend uses the fastly-protected public url for drupal, we unnecessarily generate
fastly traffic (since drupal also runs behind fastly, as for our normal procedures), and we
might incur in issues with rate limiting (since the frontend is seen as an individual "client"
doing a lot of calls, and it might be blocked,

[Zionknight-crypto]

Looks good to me, not the greatest at configs but this looks clear enough. Where exactly do we retrieve and see these internal urls? It would be great to document this with a short readme eventually.

[vermario]

I found this issue in tests:

basically the email link is generated using the internal url for drupal this way... will have to fix

[vermario]

I found this issue in tests:

basically the email link is generated using the internal url for drupal this way... will have to fix

Fixed in b5cfb18

[joshua-scott]

This approach here is now working nicely on a live site too! A few comments from that implementation to consider, then this should be good to go

[joshua-scott]

In lando also just http://appserver_nginx should work, I don't think the .nextdrupalstarterkit.internal part is needed. I suppose it's one less thing that would need to be changed later

[joshua-scott]

Done in 339c79a

[vermario]

hi! unfortunately with this change the lando setup did not work anymore (could not get data from the backend). I reverted it in 4fc3bfb and now it works again.

[joshua-scott]

In another project, we kept logic for this in one place like this:

remotePatterns: [
    process.env.DRUPAL_BASE_URL_INTERNAL_IMAGES,
    process.env.NEXT_PUBLIC_DRUPAL_BASE_URL,
  ].map((url = "") => {
    const [protocol, hostname] = url.split("://");
    if (!hostname || (protocol !== "https" && protocol !== "http")) {
      throw new Error(`Invalid images URL "${url}" in next.config.ts`);
    }
    return {
      protocol,
      hostname,
      pathname: "**",
    };
  }),

[joshua-scott]

Done in 70bd4b6

[joshua-scott]

I think here we should actually use NEXT_PUBLIC_DRUPAL_BASE_URL, or are we 100% sure this component will only ever load from the server?

In another project (pages router but otherwise very similar) where this line in FormattedText caused an error "Attempted to access a server-side environmental variable on the client", so had to change this one

[joshua-scott]

Done in 035ebbe