Linking Databases, part 2

Posted in

#26 by Ileca
2019-08-26 at 11:36
< report >I just don't get what's the appeal of this website ( The couple of times I clicked on those links, I saw nothing of value but a very stripped-down page with a synopsis, three tags and few releases info.
I looked a little closer: is it because they host OELVN? I clicked on every VN listed under 2018 (not hard, they were only 14) and none of them was approved. Going back by clicking on every first three VN of each year, I had to reach 2012 to find something approved.
I am not putting them on trial, I mean, if someone here tells me he uses them to get hard to find nanoreno games, I guess they can stay, and I am sure you had a reason for adding them here in the first place.

Pixiv for staff? It is a Wikidata property, suppose we could import that.
Yes, for staff. I am sure you will find tons of pixiv links here if you check like you did for itchio.
I am not used to that wikidata thing but without a way to import from vndb, I fear it will dampen anyone from adding links. I mean, I have no idea how you do that and I guess I could take a look but if for adding a single pixiv link I will have to (read the one kilometer doc and) create a... what? a wikidata item? and then fill it with links? This is way too much convoluted just to add a link here.
Maybe it's easy but apart from regulars, I doubt anyone will take the time to feed a third party website.

Producers could benefit from twitter links now that I think about it.

As for erogetrailer, I was thinking about linking them to the vn page but that's the same problem we have with EGS. Very little benefit from adding them to releases but not designed for a vn page.
#27 by Yorhel
2019-08-26 at 12:11
< report >You don't have to see the point of each and every field we have in the database. Links make cross-referencing stuff much easier, so even if the site doesn't have much to add, the link itself may still have value. not actively being maintained isn't a problem for now, they still have their old content.

Note that adding links to Wikidata is really easy when the Wikidata entry already exists - and they do for quite a lot of staff entries. You don't even need an account. It's indeed a bit more involved when Wikidata doesn't have an entry yet.

I am sure you will find tons of pixiv links here if you check like you did for itchio.
But you're right about this one. 860 links. I was hoping we could avoid another large edit swarm, but meh.
#28 by beliar
2019-08-26 at 15:16
< report >I see that now the store links are displayed on the VN pages. A very welcome change. However I have also noticed that the links are not shown on the producer page.

For example if i go to Edelweiss I can see the new store links, but they are missing from the MangaGamer page.

@Ileca: I have encountered quite a few situations, where the VN download link on is the only working link left that I can find on internet, hence I still consider the site valuable, despite not being maintained.
#29 by gvbn
2019-08-26 at 16:14
< report >Is there really any point in linking to non-DL Getchu pages? In most cases the physical release has been out of stock for years and years and the page just links to the DL Getchu page instead.
#30 by Yorhel
2019-08-27 at 09:53
< report >
However I have also noticed that the links are not shown on the producer page.

Also added Twitter links from Wikidata to producer pages. We don't have all that many links in the notes field, anyway.

So this is what's left to do?
- Releases: EGS & ErogeTrailers
- Staff: Pixiv

(Well and I had some ideas for character links, but I'll defer that to when I've found a way to link that automatically - far too much work to do manually...)
#31 by alto
2019-08-27 at 21:00
< report >Love it! The recent changes and additions have been stellar :) does have a number of physical edition doujin VNs e.g. link. Probably not worth adding though.

I second Pixiv for staff, it would be super useful.

Minor request: Is it possible to exclude webshop links from the edit form when you copy a release? They are almost guaranteed to be different and not having to clear them all would be nice.
#32 by savagetiger
2019-08-27 at 21:18
< report >Stellaworth, Booth links?
Though I guess at this rate it's a bit too much.
Maybe if the list is getting too long it could be on a separate tab.
#33 by trickzzter
2019-08-28 at 14:02
< report >This entry is locked for editing: r56552
Steam AppID: 303310
MangaGamer Product Code: 143
#34 by beliar
2019-08-28 at 15:45
< report >^Added the links.
#35 by Yorhel
2019-08-29 at 10:46
< report >
Is it possible to exclude webshop links from the edit form when you copy a release?
#36 by Yorhel
2019-09-01 at 10:44
< report >
- Releases: EGS & ErogeTrailers
Done. I expect that these will take a year or two to fill to some sort of completion. Take your time~

I noticed that ErogeTrailers has links to EGS and shops and has something of an API, so we might be able to use that to automate some edits at some point. But we'll need the links first, so, uh, not anytime soon.Last modified on 1970-01-01 at 00:00
#37 by gvbn
2019-09-01 at 11:49
< report >If a release is available on both the doujin and the pro sections of the same site, which one to link to?

e.g. Taimanin Asagi Zero

Gyutto doujin: link
Gyutto pro: link

DMM doujin: link
DMM pro: link

Right now, Gyutto pro version gives an extra 5% in points back over the doujin version, so they can be subject to different sales, campaigns and coupon validity.Last modified on 2019-09-01 at 11:58
#38 by Yorhel
2019-09-01 at 11:58
< report >Uh, that's a thing? And they're the exact same release? Does that happen often?
#39 by gvbn
2019-09-01 at 12:51
< report >Lilith, Norn/Miel/Cybele, Gyuunyuu Soft come to mind.

Norn/Miel/Cybele in particular has over 200 games released as both doujin and pro on Gyutto and DMM.
#40 by Yorhel
2019-09-01 at 13:59
< report >Gah. The correct answer is that we should accept both links at the same time. I'll see if I can figure out how to do that nicely.
#41 by Yorhel
2019-09-02 at 11:11
< report >Done, it's now possible to add multiple Gyutto & DMM links to a release.

And in case anyone's looking for work, there are still many releases with DLsite links in the notes field.
#42 by alto
2019-09-02 at 19:51
< report >
Awesome! I see it also now excludes GTIN and CAT too which makes complete sense.

The shops section by the VN that displays prices shows both prices from releases with multiple VNs and a single VN. Do we want both? Could be confusing if a VN doesn't have a price for just itself yet (hello 50 VN publisher packs) but I've seen a few old VNs that you can only buy together with other VNs.Last modified on 2019-09-02 at 19:55
#43 by kumiko1
2019-09-03 at 11:53
< report >Can we get EGS links for staff and producers as well (and maybe characters too, although they're not very comprehensive on EGS)

Also could you maybe automate adding EGS entries, at least for everything where the original name matches the EGS name exactly.
#44 by Yorhel
2019-09-03 at 11:56
< report >
- Staff: Pixiv
Also done. I did my best to train Multi to extract the Pixiv IDs from the notes, but with ~800 entries and many different formats, that was an incredible challenge. She's a terrible student, too. I just hope she didn't mess up too badly.

The shops section by the VN
I'll see if I can tweak that. It should show both, but there should be a way to make it clear that the link includes more than just that VN. Partials have a similar problem, not entirely sure how to handle that yet.

Also could you maybe automate adding EGS entries
I can't crawl EGS. ¯\_(ツ)_/¯

More links: Maybe... but I also think it's time for a little break in this area. :)Last modified on 2019-09-03 at 12:00
#45 by kumiko1
2019-09-03 at 12:22
< report >You don't need to crawl EGS, you just need a table of all IDs and gamenames in their database, right?

#46 by Yorhel
2019-09-03 at 12:46
< report >Managed to match 7581 entries that way, but no idea if these are correct. Full list: link (warning: large page, sloooow).

I'm seeing many releases with multiple EGS links. Is that a thing or just bad matching?

EDIT: Cleaner lists matching only non-deleted japanese releases: 6683 matches now, ordered by release ID and ordered by EGS id.Last modified on 2019-09-03 at 12:56
#47 by kumiko1
2019-09-03 at 14:24
< report >The duplicates are either different games with identical names like link and link, or different releases of the same game, like link PS2 and link PC

You could maybe just automatically add everything with no conflicting matches and do the other ones manually, since for most of the games this looks fine
#48 by beliar
2019-09-05 at 21:25
< report >GameJolt also has a surprising number of visual novels (I would say more than GOG does), so it could also be added as a dedicated field. And I still think that Nutaku does deserve a field, even if the company itself has a fair share of controversies.

Moreover, if you decide to create a dedicated field for Patreon, it shouldn't be solely done for producers, but also for releases, as many VNs actually have Patreon as their singular homepage.

With the number of dedicated fields having increased, maybe it would be a good idea to make the whole "Links" section of releases expandable. Say, put an arrow symbol under links, and it would show all the fields if pressed on. That way it won't take such a large stretch of the editing page and you will only expand it if you actually have any links to add.
#49 by Yorhel
2019-09-08 at 09:49
< report >@kumiko1: Is there a similar dump that includes release dates? Because I keep finding quite a bunch of bad matches (many of them the wrong platform and a few completely different games).

GameJolt + Nutaku
Noted for the next batch; Though I think we're fine with the current selection for the moment.

but also for releases, as many VNs actually have Patreon as their singular homepage.
In that case there's not much value in having it as dedicated field, as the website field covers it already. The main reason for adding separate fields is when an entry has multiple links.Last modified on 1970-01-01 at 00:00
#50 by kumiko1
2019-09-11 at 04:40
< report >Here: link

For future reference you can get these yourself by using EGS's custom SQL form, form link table name list link

Although release date + name should be enough to get rid of almost all of the bad matches I think? You'll still miss out on the vndb release names with the edition name in them though, so might want to match on vn original title + release date?