API: expose more information?

Posted in

#26 by db-ident-1
2022-10-08 at 02:38
< report >Since we are doing HTTP, and can (of course) benefit from Gzip in responses, I would recommend that we represent tags in VNs with objects instead of an array with 3 numbers, which can become awkward to deserialize to objects in languages such as Java and Dart (I had to convert tag IDs to float, which I do not really want to do), and I constantly forget which number represents what...
#27 by Yorhel
2022-10-08 at 14:40
< report >Tuples are a pretty common concept in many languages, not being able to easily decode their obvious JSON counterpart seems like weird oversight. :/
That said, using objects for tags is certainly an improvement, with the field selection thing it's now possible to provide extra information for those who want it, without incurring an extra cost for clients that don't need it.

I've added a bunch more fields to the VN querying API, which is looking mostly complete now.
#28 by db-ident-1
2022-10-12 at 02:27
< report >(Coming back after a few days of playing with Stable Diffusion)
Can I throw any release filter into a VN filter's "release" field? If that is the case it is much easier to write the components.

BTW, can I rely on the fact that this site has HTTP2 enabled? Dart's HTTP client situation is a little bit... weird, and a mixture of H1 and H2 can cause some problems. I will replace the network stack with OkHttp or Chrome's cronet (so it will not be a problem on Android devices), but still need the Dart stack to run on Windows.
#29 by Yorhel
2022-10-12 at 05:49
< report >
Can I throw any release filter into a VN filter's "release" field?
Yes. The only limitation is that you can't recursively nest filters of the same type. Releases have a "vn" filter, but you can't go VN -> release -> VN.
(I haven't implemented this restriction on field selection yet, that does allow infinite recursion at the moment...)

EDIT: Nevermind, the server does support infinite recursion in filters, the limitation only exists in the web UI.

BTW, can I rely on the fact that this site has HTTP2 enabled?
Eeeeh, I probably won't be disabling HTTP2 in the near future, but I'm also not sure I want to commit to supporting it indefinitely. If you must choose, better stick to HTTP/1.1 to be on the safe side.Last modified on 2022-10-12 at 06:07
#30 by db-ident-1
2022-10-12 at 07:25
< report >Well, that should be fine, since all the intended platforms (Android, iOS, Web) all use more mature network stack that supports almost everything. It is Dart's default stack that is problematic.
#31 by db-ident-1
2022-10-15 at 04:56
< report >I would like to request stats to be included in the API so they can be shown to the user in the list, just like the web UI.
#32 by Yorhel
2022-10-15 at 06:09
< report >Done.
#33Post deleted: There's many ways to promote open source and values, but rants like these aren't productive.
#34 by db-ident-1
2022-10-17 at 02:31
< report >Is it possible to enable CORS on the APIs so that other sites can make requests? You will need to respond to HEAD requests (preflights) and respond with correct headers (especially Access-Control-Allow-Origin and Access-Control-Max-Age). I am planning to deploy the web version.

For more info, see link.Last modified on 2022-10-17 at 02:31
#35 by Yorhel
2022-10-17 at 05:54
< report >That should already be enabled, but I haven't tested actual browser requests yet. Preflights are OPTION and regular API calls only respond with a CORS header if the request includes an 'Origin' header.
#36 by db-ident-1
2022-10-18 at 04:48
< report >CORS looks fine and my web version is already up. However, is it possible to make information of tags and traits available through the API (or at least allow CORS on the dumps site)?

I am currently embedding a copy of recent tag and trait dump in my app, which requires frequent updating or the user will not be able to see the description of new tags.
#37 by Yorhel
2022-10-18 at 10:28
< report >
is it possible to make information of tags and traits available through the API
Done, kind of. Still trying to figure out the best way to provide information about the tag/trait hierarchies.
#38 by db-ident-1
2022-10-20 at 15:01
< report >What is the threshold used on the web interface for image{sexual,violence}?
#39 by Yorhel
2022-10-20 at 17:50
< report >0.4 and 1.3.
#40 by db-ident-1
2022-10-21 at 01:56
< report >What's the status of language "Chinese"? It is not searchable and can not be added as title, but present in title preferences in the settings page. Selecting it does not really work since we only have zh-Hans and zh-Hant in editing UI.

I was half expecting "zh" to be "zh-Hans or zh-Hant", but this does not seem to be the case.
#41 by Yorhel
2022-10-21 at 05:03
< report >"Chinese" is still used for staff & producer entries, but since title preferences don't apply to those yet, the setting currently doesn't do a thing. I've been meaning to fix that some time™.
#42 by db-ident-1
2022-11-04 at 15:44
< report >I noticed in the database that many external links (e.g. Steam App ID) are actually stored in an numerical format, is it possible to query these values (or at least make them available through the API)?

I just had the (maybe bad) idea of a script that matches one's Steam game list to VNDB entries. Maybe this is better left to the query interface?

console.log("SELECT id FROM releases WHERE l_steam IN (" + rgGames.map((x)=>x.appid).join(',') + ")")

BTW, the link to SQLPad's homepage is dead.Last modified on 2022-11-04 at 15:45
#43 by Yorhel
2022-11-18 at 10:25
< report >I've added an 'extlink' filter and 'extlinks' field to release entries. The former isn't too performant yet, but that can be easily solved with an index. The latter isn't really efficient for the "match steam IDs" use case, but at least the information is available. I may add separate direct-id fields later on if that turns out to be worth it. Matching with external libraries sounds pretty useful, in any case.


You must be logged in to reply to this thread.