Some love for the API

Posted in

#1 by Yorhel
2013-01-04 at 19:35
< report >Just a quick note to remind developers of the fact that we actually have an API, and to announce a few updates.

The TCP API (d11) has been updated to allow list managent - fetching and updating of your votes, VN list and wishlist - and you can now get the list of tags for each VN as well.

I've also added a database dump for the tags (d14), as that information doesn't change all that often and I figured it'd be easier and more efficient to just provide that data in bulk than to add all kinds of querying operations to the TCP API.

As always, if you're working on an application and want to interact with the database with functionality not yet provided by the API, don't hesitate to contant me and tell me what you need. Also contact me if the stuff you want *is* provided by the database but you're wondering about the most efficient way get it. I may be able to provide some new insight or optimize the API where needed.

And please don't crawl VNDB! Even if it may look tempting, I occasionally firewall people who ignore that warning.Last modified on 1970-01-01 at 00:00
#2 by yirba
2013-01-04 at 23:12
< report >Oh, I'm quite interested in the new feature. Might have a slight play around with it sometime…
#3 by kelpie
2013-01-05 at 19:35
< report >There's no way to filter on tags with the API, right? I see that the dump doesn't actually list which VNs have a given tag, and d11 doesn't say that you can filter on tags. The question is mostly academic, since the search interface on the web site does pretty much whatever I need. The vnlist and wishlist features should prove interesting, though.
#4 by Yorhel
2013-01-05 at 20:05
< report >There's currently no way search VNs by tags, no. I don't really feel like implementing functionality when I've no idea whether and how it's going to be used. It's much easier to add features when there's an actual use case. :)
#5 by Yorhel
2013-01-06 at 09:31
< report >I see that some people use multiple 'get vn .. (id=..)' commands to fetch information about a list of VNs. That's fine, but you're more likely to hit the throttle error that way. A slightly more efficient approach is to use the array filter on the VN id, as such:
get vn .. (id=[5,7,17,..]) {"results":25}
With that, you can fetch information of up to 25 VNs with a single command.Last modified on 2013-01-06 at 09:34
#6 by wannabiteme
2013-01-09 at 20:16
< report >It sounds great.
As I lack of big skills in "internet programming" May I ask for some 'copy-paste' examples that I can use immediately? I just need to know where to put what and what language to use(javascript maybe?).
Thank you in advance.
#7 by Yorhel
2013-01-09 at 20:35
< report >"internet programming" is the same as regular programming, just with different libraries. The API is rather useless if you're not already familiar with programming. >_>

And you can use pretty much any language. But if you want to use Javascript you'll need to use Node.js or something similar, since the browser does not provide a TCP API. (Except for Chrome extensions, maybe. I've no idea how those work, though)Last modified on 2013-01-09 at 20:36
#8 by yirba
2013-01-10 at 00:47
< report >@6: It really depends on what sort of program you want to create. I expect most people would use it for web applications, in which case you'll probably want to have something that runs server-side (although I /guess/ you could use Java applets or Flash embeds, that would probably be quite messy). If that's the case, languages such as PHP, Python, Perl or Ruby would be the way to go. Each language will have their own libraries for handling TCP connections, so you'll probably want to check with the programming language's documentation to find out how to do this.
#9 by wannabiteme
2013-01-12 at 01:17
< report >@7
I'm familiar with C-like languages. Well... node.js. I should check it.
I wanted to do something like 'mal updater': login, check your list, search for particular visual novel, add, update...something like that.
Python & Ruby - might be good for me too, thanks for that.
ps. Fast answer. Thank you.
#10 by kelpie
2013-01-12 at 06:53
< report >Did you change the connection time out? I could have sworn that connections stayed open for hours before, but they seem to close much more quickly, now. If not, the problem is probably on my end--I haven't investigated thoroughly.
#11 by Yorhel
2013-01-12 at 08:39
< report >@wannabiteme: Some existing open source implementations that I know of (may be helpful):
- vinfo.c - A small VN info viewer in C
- Serika - A handy GUI to play around with the API (written in OCaml)
- pyvndb - A Python library
- wMAL - A multi-site list management script (Python)

@kelpie: I've not changed a thing to my code, but it's possible that some library upgrades on my side caused it, or that the new server connection is just worse. I did check the libraries I use a bit, but couldn't quickly see a reason why it'd suddenly timeout the connection.
#12 by wannabiteme
2013-01-14 at 17:33
< report >@Yorhel
That will be very helpful, I guess. Thank you again.Last modified on 2013-01-14 at 17:33
#13 by conanak99
2013-01-16 at 12:24
< report >Do the API working right now? I'm trying to implement the API using Java and C#.
I can connect to the host name "" on port 19534. But when I sent the login command and a "get vn" command, the tcp connect doesn't response anything result.
#14 by Yorhel
2013-01-16 at 12:31
< report >@conanak99: Yes, it's up and running. The most common problem when you don't see any response is usually that you forgot to append the end-of-message character (hex 4) to the outgoing command, or your code to read back the replies doesn't stop after receiving this character. I see that you've already successfully sent a 'login' command (according to my logs), so if you're not seeing a reply there's probably a bug with your reader code.
#15 by conanak99
2013-01-16 at 13:13
< report >Thank you very much. I forgot to add the EOT character. Now the API is working fine.Last modified on 2013-01-16 at 13:13
#16 by space-ranger
2013-01-17 at 21:39
< report >I tried playing a little bit with the API and I came up with a perl script to add to the list of example codes.
the script

It turned out to that it wasn't that tricky to do. The ability to turn JSON data structures into perl hashes and arrays with a single instruction is pretty neat and it's quite useful when handling the responses.Last modified on 2013-01-17 at 21:40
#17 by Yorhel
2013-01-18 at 14:28
< report >Added support for 'get character' (d11#4.4) and a dump of the traits database (d14#3). I got bored halfway, so the characters API is still lacking some features such as VN, releases and instance information and VN/trait filters. Just bug me if you want that functionality, too.
#18 by dokugogagoji
2013-02-14 at 18:34
< report >Really nice...I should try to make a viewer for my phone...
#19 by yirba
2013-03-10 at 21:39
< report >I've been having a bit of a problem with the API. I'm not sure about the syntax for multiple arguments?

get vnlist basic ((uid=0) and (page=1))
error {"msg":"Unknown field 'page'","value":1,"id":"filter","op":"=","field":"page"}

EDIT: No worries; figured it out. :-)

Should be:

get vnlist basic (uid=0) {"page": 1}
Last modified on 2013-03-10 at 21:42
#20 by diffycat
2013-05-13 at 18:57
< report >link - java api library

Can you please add a method to get current api version? (to check library compatibility before doing anything)
#21 by Yorhel
2013-05-13 at 19:46
< report >@diffycat: Nice. Looks pretty complete, too.

Regarding api version: I'm not sure what purpose it'd serve. You tell the API which version you support in the login command, and the server will provide that version of the API even if I make some incompatible changes in the future. There are no issues that a further check in the library might solve.Last modified on 2013-05-14 at 20:15
#22 by maraskanuser
2013-12-29 at 02:35
< report >There is no way to ask for a VN's bayesian rating through the API, yet. Would it be feasible to implement this, considering server load and so on?Last modified on 2014-01-29 at 23:27
#23 by z411-main
2014-08-03 at 18:07
< report >Very nice work yorhel, it works great.
I don't know if the vnlist/votelist tables store a last updated timestamp, but if they do, load a filter to only get items updated after a certain time/date would be useful.
#24 by Yorhel
2014-08-04 at 05:14
< report >
I don't know if the vnlist/votelist tables store a last updated timestamp
They don't.
#25 by ravosu
2014-08-19 at 01:20
< report >I don't know if this is the right place to ask, but neither or ( let me connect to 19534.

The first IP just gets stuck at connect() until I get a ETIMEDOUT, and the second IP just refuses the connection to that port.

Did the connection information change? Thanks!