Limits on Mender Inventory Size

nomtor0204 · December 12, 2024, 2:56am

Hello,

Are there any limits on the size of inventories that can be sent to the Mender server?

On one particular device, I encountered an issue where an inventory with tens of thousands of characters could not be sent. For example:

key_name=AAAAAAAAAA...AAA (tens of thousands of characters)

I would like to confirm if there are any specifications regarding inventory size.

I reviewed the Inventory document, but we did not find any information about inventory size limits.

Any guidance on this matter would be greatly appreciated. Thank you very much.

TheYoctoJester · December 12, 2024, 2:56pm

Hi @nomtor0204,

The scope of a key-value pair is currently limited to 1024 characters. Is there a specific use case that you want to tackle?

Greetz,
Josef

nomtor0204 · December 17, 2024, 6:27am

Hi @TheYoctoJester ,

Upon investigation, it appears that in Mender-client 3, the buffer size used for parsing inventories is set to 64KB, effectively limiting the size of inventory data that can be sent to 64KB.

The following source code shows that the buffer created by bufio.NewScanner is 64KB:

github.com

mendersoftware/mender/blob/0e326002fc2d013cd7f44650b99068cbddf83540/utils/kvparser.go#L37


      
          // merged into a single list.
          type KeyValParser struct {
          	data map[string][]string
          }
          
          func (k *KeyValParser) Parse(raw io.Reader) error {
          	if k.data == nil {
          		k.data = map[string][]string{}
          	}
          
          	in := bufio.NewScanner(raw)
          
          	for in.Scan() {
          		if err := in.Err(); err != nil {
          			return errors.Wrapf(err, "failed to read input line")
          		}
          		line := in.Text()
          
          		if len(line) == 0 {
          			continue
          		}

When calling the Mender API’s Assign Attributes directly or using Mender-client 4, it was possible to send inventories larger than 64KB. This suggests that the 64KB restriction applies only to Mender-client 3.

In our case, we needed to send inventories exceeding 64KB using Mender-client 3. We addressed this by patching the Mender-client to increase the allowable inventory size.

Just wanted to let you know as a quick update.

Best Regards,
Ryo

Topic		Replies	Views
Update Inventory from Client General Discussions	4	194	November 1, 2023
Can a mender-client inventory attribute value be a json file? General Discussions	2	232	June 30, 2021
Upload Artifact fails because of size General Discussions	6	1453	October 6, 2021
How to upload large file(In GBs) to mender server using mender APIs? General Discussions mender-server	1	59	June 28, 2024
Mender enterprise - client cannot update inventory General Discussions	8	1625	November 17, 2020

Limits on Mender Inventory Size

Related topics