How does one go about allowing link embeds on services like Discord?

[GreemDev]

Potentially related: discord/discord-api-docs#6385

Information about my installation:

.deb installation; ran using the provided systemd units in the repository.

Image previews don't work:

and

(tried sending multiple times because I thought the issue could be #1060, but it still never embeds)

I looked around the documentation and I couldn't find what I wanted.
I know a configuration resembling my desired outcome is possible; embeds from Codeberg work flawlessly as do image previews in webhook notifcations, and Codeberg uses Anubis as well.
Any help would be immensely appreciated!

Additionally, thank you for making this awesome tool

[GreemDev]

Got it.

  - name: user-avatars
    path_regex: ^/avatars/.*$
    action: ALLOW

  - name: discord-embeds
    user_agent_regex: ^Mozilla/5\.0 \(compatible; Discordbot/2\.0; \+https://discordapp.com\)$
    action: ALLOW

[Aerodymeus]

Not sure if it a fine method to play allow it in the botPolicy.yaml, but in the end it helped me as well for getting the embedding back.

[bctrainers]

Had a quick peek at this, decided to see what swaths of IP ranges/CIDRs that are in use for Discord's embed service/meta-scraper. From quick testing this morning, it would appear it's utilizing a decent swath of Google Cloud's ranges. Unfortunately, many of the IP's contained within the following CIDRs have been observed in years past getting utilized for probing/scraping at random intervals. Which would ultimately mean that these are spot instances getting used and not a static IP range owned-by/leased-to Discord.

• 35.192.0.0/12
• 35.208.0.0/12
• 35.224.0.0/12
• 35.240.0.0/13

The items above were some of the CIDR's being seen on my server logs when throwing random URL's into a channel. All having the same UA as what you've depicted: Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com).

With that said, it might be safe to assume that Discord is using the entire NA+EU Google Cloud IP ranges available to spin up instances with random IPs as required for various needs.

[avioletheart]

this isn't specified publicly but discord sets rDNS as *.discord.com on Discordbot UA requests, that's how they want you to validate requests.

[bctrainers]

this isn't specified publicly but discord sets rDNS as *.discord.com on Discordbot UA requests, that's how they want you to validate requests.

Very interesting. That would make it much more sane when doing a validation check via Anubis then! Thanks for the insight.

[LeviSnoot]

Since validating Google Cloud spot ranges can be a bit dicey, I thought I'd make use of https://cdn.discordapp.com/ipranges/discord.json by periodically fetching it and writing the ip range to a "known good" rule for Anubis to consume.

#!/usr/bin/env python3
import json
import urllib.request
from pathlib import Path

OUT = Path('/etc/anubis/known-good/discord.yaml')
SOURCE_URL = 'https://cdn.discordapp.com/ipranges/discord.json'

HEADER = """# Generated from https://cdn.discordapp.com/ipranges/discord.json
"""

RULE_TEMPLATE = """- name: discord-embeds
  user_agent_regex: ^Mozilla/5\\.0 \\(compatible; Discordbot/2\\.0; \\+https://discordapp\\.com\\)$
  remote_addresses:
{items}
  action: ALLOW
"""


def fetch_prefixes():
    req = urllib.request.Request(
        SOURCE_URL,
        headers={'User-Agent': 'Mozilla/5.0 (compatible; anubis-discord-ranges/1.0)'}
    )
    with urllib.request.urlopen(req, timeout=15) as r:
        data = json.load(r)
    return [p['ipv4Prefix'] for p in data.get('prefixes', []) if 'ipv4Prefix' in p]

def render(prefixes):
    items = '\n'.join(f'    - {p}' for p in prefixes)
    return HEADER + RULE_TEMPLATE.format(items=items)

def main():
    prefixes = fetch_prefixes()
    if not prefixes:
        raise SystemExit('No IPv4 prefixes returned by Discord API')

    new_text = render(prefixes)

    if OUT.exists() and OUT.read_text() == new_text:
        print('discord.yaml: no change')
        return

    OUT.parent.mkdir(parents=True, exist_ok=True)
    tmp = OUT.with_suffix('.tmp')
    tmp.write_text(new_text)
    tmp.replace(OUT)

    print(f'Wrote {len(prefixes)} prefixes to {OUT}')

if __name__ == '__main__':
    main()

Set up a cronjob for this and import the rule:

- import: /etc/anubis/known-good/discord.yaml

This creates:

# Generated from https://cdn.discordapp.com/ipranges/discord.json
- name: discord-embeds
  user_agent_regex: ^Mozilla/5\.0 \(compatible; Discordbot/2\.0; \+https://discordapp\.com\)$
  remote_addresses:
    - 104.196.222.45/32
    - 104.196.23.152/32
    - 34.138.20.118/32
    - 34.138.218.50/32
    - 34.139.115.226/32
    - 34.148.171.212/32
    - 34.73.62.161/32
    - 34.74.100.84/32
    - 34.74.113.139/32
    - 34.75.19.245/32
    - 35.190.130.193/32
    - 35.196.132.85/32
    - 35.227.34.53/32
    - 35.227.62.178/32
    - 35.231.179.220/32
    - 35.231.24.24/32
    - 35.237.169.55/32
    - 35.237.4.214/32
    - 35.237.58.227/32
    - 35.243.226.206/32
  action: ALLOW

Obviously do not blindly copy the rule above, it's just an example and discord's IP ranges could change at any time. Use the script if you're going to set this up.