batch::get-many should parallel store.get

[devigned]

get: func(key: string) -> result<option<list<u8>>, error>;

in store returns an option<list<u8>>, but

get-many: func(...) -> result<list<option<tuple<string, list<u8>>>>, error>;)

in batch returns list<option<tuple<string, list<u8>>>>. I would expect get-many to return result<list<tuple<string, option<list<u8>>>>, error>;.

[Mossaka]

this makes sense to me.

Since this updates the APIs, can you bump the draft version up?

[thomastaylor312]

I can go either way on this. I think we may have had it the way you have it in this PR before, but I swapped it out because you have to return a copy of the key. For a batch request, that means even if you have no data to show, you have to return a new key (which could be quite large for some people). Since batch requests are big, this is where extra allocations of keys could matter. I swapped it to the option because it should be returned in the same order as passed in, so if you absolutely need to know which key you need, you can refer to that. Here are our options and the tradeoffs:

What you have in this PR:

list<tuple<string, option<list<u8>>>>

Advantages: Easiest to consume because you have each KV pair
Disadvantages: Extra allocation of every key that is passed in

Current:

list<option<tuple<string, list<u8>>>>

Advantages: No extra allocations of non-existent keys
Disadvantages: Awkward mapping back to initial list of keys if you need to know the missing key

Option 3

list<option<list<u8>>>

Advantages: No extra allocation of keys
Disadvantages: Required iterate over the list of keys and results together (which can be awkward/annoying in some languages)

So I think those are the options. After putting some thought into it here, I'd rather have option 3 to avoid all sorts of extra memory usage (and copying), but we should decide together what we'd prefer here

[ChihweiLHBird]

@thomastaylor312 if we want "no extra allocations of non-existent keys", I think list<tuple<string, list<u8>>> would be a better option, where all non-existent keys are simply dropped. Or list<tuple<string, option<list<u8>>>> (this PR) is good too if we want to handle the case of data source (e.g., kv db) storing the key with an explicit value of null. We can then specify that all non-existent keys must be dropped and (key, none) entry can only be used for an explicit null.

I think the current one is the worst, because I can't think of any use case of having None in the list, list{None, None, None, ("my_key", "my_value")}

[ChihweiLHBird]

Hi @danbugs @devigned @Mossaka, can we reconsider this PR for merging?

[danbugs]

Updated the doc comment as requested and also updated the wit-abi-up-to-date tooling version to get a passing CI. LGTM.