Skip to content

HDDS-15481. Avoid re-encoding the URL safe token for each read chunk#10433

Open
sodonnel wants to merge 1 commit into
apache:masterfrom
sodonnel:HDDS-15481-token-encode
Open

HDDS-15481. Avoid re-encoding the URL safe token for each read chunk#10433
sodonnel wants to merge 1 commit into
apache:masterfrom
sodonnel:HDDS-15481-token-encode

Conversation

@sodonnel
Copy link
Copy Markdown
Contributor

@sodonnel sodonnel commented Jun 4, 2026

What changes were proposed in this pull request?

This is a small performance improvement - the Token is re-encoded as a URL-safe base64 string on every call to readBlockImpl(), which will be called for each 1MB read.

We can cache the URL encoded value and on re-encode it if the token is refreshed.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15481

How was this patch tested?

Depends on existing tests.

@szetszwo
Copy link
Copy Markdown
Contributor

szetszwo commented Jun 4, 2026

@sodonnel , this is a good catch! This problem was copied from BlockInputStream. How about we also fix BlockInputStream here? We should just change the token class from Token<OzoneBlockTokenIdentifier> to String in all the places.

@sodonnel
Copy link
Copy Markdown
Contributor Author

sodonnel commented Jun 4, 2026

We should just change the token class from Token to String in all the places.

How far do you think we should go? Token is used all over the place - intellij just gives up searching for usages.

The nicest fix would be to cache the encoded token inside the token object itself, but its a hadoop class, so we cannot easily change it just in Ozone.

@szetszwo
Copy link
Copy Markdown
Contributor

szetszwo commented Jun 4, 2026

Just BlockInputStream/StreamBlockInputStream. I tried to change the code and the change seems not big. The reason to involve BlockInputStream here is that they have some shared code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants