Some love for the API

Posted in

#151 by yorhel
2016-09-11 at 17:38
< report >VN relations official flag added. Deferring staff until I've figured out how to deal with the aliasses. The staff database is slightly unusual in that aspect, so I can't just copy the approach from other DB entries.

last part of a screenshot's url is its unique identifier?
"" (89357)
Yes. It's also the same number you see in the diffs of VN pages (e.g. Chicchakunai Mon! ~School Bus de Omukae Chu Chu♥~) and the "Screenshot #" on the VN edit form.

store them in 100 folders with the last 2 parts of url.
To be exact: That's the modulo 100 of the screenshot number. You can take the modulo with any other number in order to get more or less than 100 dirs.
#152 by zolty
2016-09-12 at 23:00
< report >Thanks for taking our suggestions, I have just published a new version of my application (link and discussion at t8111.8).

I don't think I will add any major features until the staff database is added to the API but there's still a few things that I want to do (like letting users set VN notes using the Userlist notes field).

Maybe in the future I could recreate the application as a website but I'm not sure how much trouble that would be.
#153 by zolty
2016-09-18 at 17:43
< report >@yorhel
Some people are having issues authenticating my client (exception is "A call to SSPI failed") and I assume that it's because authentication is more complex than simply:
(C#) sslStream.AuthenticateAsClient(VndbHost);

That's how it seems to be in most applications that I've seen in this thread, including the one that I used for the connection on my app.

I've been searching online for a tutorial and whatnot but most tutorials are for server apps, rather than clients apps.
It seems that each machine needs to have a certificate installed and use it to connect to VNDB but where do I get that?

If you could point me in the right direction, that would be very helpful.
#154 by yorhel
2016-09-18 at 17:57
< report >I already googled for that a bit, but Windows programming is too alien to me. If you want to verify the API's certificate, which I guess that C# thing does, then the user needs to have the right root certificates to validate against. Which would be either the Let's Encrypt or the Identrust root certificate. I know all browsers have either of those, but I've no clue where the C# function looks for certificates.

If it turns out that neither of those root certs comes with Windows or .NET by default, then I suppose your application needs to provide it. In any case, the certs can be found at link - the ISRG Root X1 is what you'll need.

That said, the error message you're seeing is so damn generic that it could be anything. The problem could just as well be in incompatible TLS version or unsupported ciphers.
#155 by zolty
2016-09-18 at 18:21
< report >Yeah since I can't replicate the problem it is more difficult to debug, I will try some other things.

The exception mentioned above - "A call to SSPI failed" - says see inner exception which says "The Local Security Authority cannot be contacted".

I've searched about it and it just seems to be caused by a windows update (kb3163018) which would make sense since only some people are having the issue.
Unfortunately even with that information I can't fix the problem.Last modified on 2016-09-19 at 17:29
#156 by zolty
2016-11-12 at 03:24
< report >@yorhel

Is it possible to revert a bayesian rating to a mean rating by using the vote count, or is it a one-way only conversion?

I was looking through vndb's source code but couldn't really find where that calculation is made in order to reverse it.
#157 by yorhel
2016-11-12 at 06:49
< report >It's at link - it makes use of several parameters that are not readily available from outside of the database, so it's not easily reversible.
#158 by zolty
2016-12-03 at 21:39
< report >Hey yorhel, I see that you added the get user command which allows someone to get a username out of an ID, however, I'm more interested in the opposite (get ID from username), could you add that functionality?

Also, out of curiosity, is there a way for someone to add/change userlist notes without using the API?
#159 by yorhel
2016-12-04 at 07:03
< report >Username filter added.

Notes can be set in your list by checking the item and choosing "Set note" in the "-- with selected VNs --" dropdown. (That UI is awful...)
#160 by zolty
2017-01-07 at 13:11
< report >Thanks for adding the filter, would it also be possible to add a trait filter to character search?

And I might not immediately reply with thanks because I don't want to bump the thread but I am certainly thankful and using those features.
#161 by kurothing
2017-01-16 at 00:46
< report >I don't see any answer in regards to this on the Database Dumps page, nor a quick search for "dump" in this thread revealing anything.

I see in the source of the site, that you only dump it every 24 hours, but i'm curious if libraries should be limiting the downloads of tags/traits/votes to reduce the stress of the server of having the serve large-ish files (EG: The votes.gz file was upto 1.2Mb when i grabbed it)

Or is it fine to put no limitation of on methods that download the database dumps?

Also in the case of the client info sent when logging in, should it be the library implementing the api, or the application using the library? Just curious if the client name and version should be hard coded or if i should make the application implementing the library provide the details.Last modified on 2017-01-16 at 03:00
#162 by yorhel
2017-01-16 at 06:34
< report >Regarding downloading dumps: The server's fine on bandwidth, so you don't have to worry much about that. But if it's easy to cache it may still be worth doing so to optimize your users' bandwidth. :)

Regarding client name+version: Preferably, let the application set these.
#163 by zolty
2017-01-25 at 18:52
< report >#161
You can just download them every day, or is that to much data for you? (not being sarcastic, just dont know what kind of constraints you are working with)

I have dumps set to auto-update every 2 days, I don't download the votes though , tags and traits together are less than a megabyte.

In my case, the negative effects of that is people cant search for tags/traits that are less than 2 days old, which probably affects 0% of users.
#164 by yorhel
2017-03-03 at 10:43
< report >
would it also be possible to add a trait filter to character search?
Belated done.
#165 by vndbandroid
2017-04-28 at 06:19
< report >
I add stuff to the API whenever there's demand for it [...]. Adding staff is slightly more work, but that should be no problem either.

Hi, just a message to say that I am also interested in the staff API, as it seems to be a requested feature from my users.
Anyway thanks for maintaining the API, working great so far.
#166 by yorhel
2017-04-28 at 10:03
< report >Thanks for the reminder. Happened to have time, so I've added basic support for fetching staff. It's still very limited because the interaction between staff and aliases and VNs and characters is not terribly simple, so I only implemented some functionality where the matching API would be somewhat obvious.

I'll need some feedback to continue: What required functionality is missing, and what API would be practical?
#167 by vndbandroid
2017-04-28 at 11:32
< report >Alright, thanks!
I don't have time to test it right now, but from what I can see in the doc:
- I would need a way to retrieve all the VNs and characters for which a staff is credited. For example, for s607, the list of the "Credits" and "Voiced characters". It could be respectively a list of vn ids and a list of character ids in the "get staff" command. Edit: uuh... ids may not be enough actually... Well I don't know what the best way to put it here is... I don't know if you can include a list of "partial VN objects" and "partial character objects" in "get staff" with the basic info displayed on this staff page...
- Also, from a character, I would need to get the staff which voices it, so I can reproduce the "Character summary" from Kono Oozora ni, Tsubasa o Hirogete for example. Again, it can simply be a list of staff IDs in the "get character" command, as far as I'm concerned. Edit: including the staff's romaji name and gender too in this list would be more practical and save API requests actually, but meh it's up to you.

Apart from this, it looks ok and I already have enough to play with!Last modified on 2017-04-28 at 11:48
#168 by yorhel
2017-05-22 at 17:08
< report >Let me apologize for two things: The delay, and the terribly unhelpful new API. :-)

Everything you asked for is now available in the API, but I suppose it's not very convenient to query with all the IDs being thrown around. I did consider including some extra fields in the returned lists (and that option is still open), but some of those lists can grow pretty large already as it is, and I want to avoid sending super large responses. Splitting that up into multiple commands should be efficient enough, but will not simplify your code.
#169 by zolty
2017-05-23 at 17:19
< report >Thanks for the staff additions, yorhel.

This combination of commands now allows for a user to get staff from a title and then get all titles from the staff so it's quite good.

However, I would really appreciate a name filter for get staff so that I don't have to go through a 'get vn staff' command.

Also one question, the vns and voiced fields respectively correspond to credits and voiced characters in the staff pages, correct?
#170 by vndbandroid
2017-05-23 at 17:20
< report >@yorhel No problem for the delay!
Alright I took a look at the new API and it looks feasible. I'll implement a version with this and get back to you if I have any problem or performance issues (although it may take quite a bit of time for me to develop all of this, so don't expect a quick feedback... :p).Last modified on 2017-05-23 at 17:20
#171 by yorhel
2017-05-23 at 17:39
< report >
Also one question, the vns and voiced fields respectively correspond to credits and voiced characters in the staff pages, correct?
#172 by vgamer
2017-05-31 at 15:03
< report >I've got a few issues with the API, specifically the 'get release' command: First off, I can't seem to get it to return the platforms array, as I get "Unknown info flag 'platforms"' when adding it to the flags list. Secondly, it'd be nice if releases could be filtered by platform, though that's easy enough to do on the client side if necessary (which is what I'm trying to do currently).

Thanks for the great work on this website! It's a very valuable resource, and I hope it continues to grow and improve.
#173 by vgamer
2017-05-31 at 16:00
< report >Oh, my bad. I misread the documentation. If I add in the details flag I get platforms. It'd still be nice to be able to filter releases by platform though, since you can do it with VNs.
#174 by vgamer
2017-05-31 at 17:09
< report >Another suggestion: Allow the vn filter field to accept arrays of integers as well as just single values.
This would make it easier on your servers when programmers want to grab releases of multiple VNs at the same time, as currently lots of requests are required to do so.
#175 by yorhel
2017-06-21 at 06:43
< report >Added some requested features, check d11#8 for the latest changelog.