serrebi/BlindRSS

Watch

Fork 0

mirror of https://github.com/serrebidev/BlindRSS.git synced 2026-08-13 11:59:26 -07:00

Support RSS feed URLs containing non-ASCII characters #44

New issue

Closed

opened 2026-07-03 07:55:55 -07:00 by serrebi · 3 comments

serrebi commented

2026-07-03 07:55:55 -07:00

Owner

Technical Breakdown of the Issue

The problem manifests in two distinct scenarios, each requiring a specific encoding approach:

Scenario 1: Non-ASCII Characters in Domain Names

When non-ASCII characters appear in the domain name (e.g., пример.рф), the domain must be converted to its ASCII Compatible Encoding (ACE) representation, commonly known as Punycode (e.g., xn--e1afmkfd.xn--p1ai). BlindRSS currently does not perform this conversion, leading to resolution failures.

Scenario 2: Non-ASCII Characters in Path/Query Components

When non-ASCII characters are present in the path or query string of the URL (e.g., example.com/категория), these characters must be percent-encoded according to RFC 3986 (e.g., example.com/%D0%BA%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F). BlindRSS currently lacks this encoding step, resulting in invalid URL construction and failed requests.

Proposed Solution

It is proposed to implement automatic handling of non-ASCII URL components within BlindRSS, ensuring compatibility with how modern browsers resolve such addresses. The solution should cover both import workflows (OPML) and manual entry.

Expected Behavior

The following behavior is expected after implementation:

For Domain Names

BlindRSS should automatically convert any non-ASCII domain names to their Punycode representation before attempting DNS resolution.
Users should be able to input domains like пример.рф directly; the application should handle the conversion internally without requiring manual intervention.

For Path and Query Components

BlindRSS should apply proper percent-encoding to any non-ASCII characters in the path and query segments of the URL.
URLs such as example.com/категория should be transformed to their encoded form (example.com/%D0%BA%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F) before making HTTP requests.

Workflow Coverage

The encoding logic must be applied consistently across all input methods:
- Manual addition via the “Add New Channel” dialog.
- Import of feeds from OPML files.
Invalid or malformed URLs should still be rejected with a clear error message, but valid non-ASCII URLs must be processed successfully.

Additional Notes

This change should not alter the way URLs are stored or displayed to the user; the normalization should occur only at the request layer.
Compatibility with existing subscriptions should be maintained; no migration of stored URLs is required unless they were previously saved in an incorrect format.

It is required to address the inability of BlindRSS to handle RSS feed URLs that include non-ASCII characters. Currently, the application fails to process such links when they are imported via an OPML file or added manually through the “Add New Channel” dialog. This limitation prevents users from subscribing to feeds hosted on domains or paths that use non-Latin scripts. #### Technical Breakdown of the Issue The problem manifests in two distinct scenarios, each requiring a specific encoding approach: ##### Scenario 1: Non-ASCII Characters in Domain Names When non-ASCII characters appear in the domain name (e.g., `пример.рф`), the domain must be converted to its ASCII Compatible Encoding (ACE) representation, commonly known as Punycode (e.g., `xn--e1afmkfd.xn--p1ai`). BlindRSS currently does not perform this conversion, leading to resolution failures. ##### Scenario 2: Non-ASCII Characters in Path/Query Components When non-ASCII characters are present in the path or query string of the URL (e.g., `example.com/категория`), these characters must be percent-encoded according to RFC 3986 (e.g., `example.com/%D0%BA%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F`). BlindRSS currently lacks this encoding step, resulting in invalid URL construction and failed requests. #### Proposed Solution It is proposed to implement automatic handling of non-ASCII URL components within BlindRSS, ensuring compatibility with how modern browsers resolve such addresses. The solution should cover both import workflows (OPML) and manual entry. #### Expected Behavior The following behavior is expected after implementation: ##### For Domain Names - BlindRSS should automatically convert any non-ASCII domain names to their Punycode representation before attempting DNS resolution. - Users should be able to input domains like `пример.рф` directly; the application should handle the conversion internally without requiring manual intervention. ##### For Path and Query Components - BlindRSS should apply proper percent-encoding to any non-ASCII characters in the path and query segments of the URL. - URLs such as `example.com/категория` should be transformed to their encoded form (`example.com/%D0%BA%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F`) before making HTTP requests. ##### Workflow Coverage - The encoding logic must be applied consistently across all input methods: - Manual addition via the “Add New Channel” dialog. - Import of feeds from OPML files. - Invalid or malformed URLs should still be rejected with a clear error message, but valid non-ASCII URLs must be processed successfully. #### Additional Notes - This change should not alter the way URLs are stored or displayed to the user; the normalization should occur only at the request layer. - Compatibility with existing subscriptions should be maintained; no migration of stored URLs is required unless they were previously saved in an incorrect format.

serrebi closed this issue

2026-07-03 08:37:10 -07:00

serrebi commented

2026-07-03 08:37:10 -07:00

Author

Owner

Fixed in v1.77.4, covering both scenarios you described:

IDN domains: hostnames like пример.рф are converted to their Punycode form (xn--e1afmkfd.xn--p1ai) using UTS-46 mapping, matching how browsers resolve them.
Path/query components: non-ASCII characters like example.com/категория are percent-encoded per RFC 3986. Already-encoded sequences are never double-encoded.

The normalization happens at the request layer inside the shared HTTP helpers, so every workflow is covered uniformly — the Add New Channel dialog, OPML import, feed refresh, and feed discovery. As you requested, stored and displayed URLs remain exactly as entered; only the outgoing request is transformed. No migration of existing subscriptions is needed.

Thanks for the detailed technical breakdown — it made this straightforward to implement.

Fixed in [v1.77.4](https://github.com/serrebidev/BlindRSS/releases/tag/v1.77.4), covering both scenarios you described: - **IDN domains**: hostnames like `пример.рф` are converted to their Punycode form (`xn--e1afmkfd.xn--p1ai`) using UTS-46 mapping, matching how browsers resolve them. - **Path/query components**: non-ASCII characters like `example.com/категория` are percent-encoded per RFC 3986. Already-encoded sequences are never double-encoded. The normalization happens at the request layer inside the shared HTTP helpers, so every workflow is covered uniformly — the Add New Channel dialog, OPML import, feed refresh, and feed discovery. As you requested, stored and displayed URLs remain exactly as entered; only the outgoing request is transformed. No migration of existing subscriptions is needed. Thanks for the detailed technical breakdown — it made this straightforward to implement.

serrebi commented

2026-07-03 15:46:48 -07:00

Author

Owner

I’m afraid the current implementation isn’t fully working yet.

Feeds with non-ASCII characters in the URL fail to update, raising the following error:

Error: 'UnicodeEncodeError' object has no attribute 'response'

It looks like the Unicode support isn’t configured correctly at the moment.

For testing, here’s a testing feed URL that includes Cyrillic both in the domain and in the path:

https://цейковец.рф/тест.xml

It seems the issue is specifically with converting the domain name part of the URL. For comparison, here’s an analogous feed on a Latin-domain with Cyrillic in the path - it loads without errors:

https://tseykovets.ru/тест.xml

Let me know if you need any additional details or logs.

I’m afraid the current implementation isn’t fully working yet. Feeds with non-ASCII characters in the URL fail to update, raising the following error: ``` Error: 'UnicodeEncodeError' object has no attribute 'response' ``` It looks like the Unicode support isn’t configured correctly at the moment. For testing, here’s a testing feed URL that includes Cyrillic both in the domain and in the path: ``` https://цейковец.рф/тест.xml ``` It seems the issue is specifically with converting the domain name part of the URL. For comparison, here’s an analogous feed on a Latin-domain with Cyrillic in the path - it loads without errors: ``` https://tseykovets.ru/тест.xml ``` Let me know if you need any additional details or logs.

serrebi commented

2026-07-03 16:06:57 -07:00

Author

Owner

Fixed in v1.78.3. The request layer now punycodes IDN domains, percent-encodes non-ASCII paths, normalizes URL-valued headers such as Referer/Origin, and the refresh error formatter no longer assumes every exception has an HTTP response. I added regression coverage for the Cyrillic-domain feed shape from the report.