Advancing Search

Posted in

#1 by yorhel
2021-01-10 at 08:48
< report >It's getting cold outside, so I figured I'd warm myself up with a little announcement. And a large cup of hot chocolate.

I've updated the visual novel and release listings with a new advanced search system. You can now search for visual novels related to the K-On! anime, visual novels that have been unofficially ported to Windows, visual novels with both a male and female protagonist, visual novels with a female character voiced by a guy (or the other way around) and unpopular but potentially great Japanese visual novels. You can freely combine these with tags and traits filters, or filter out VNs that are already on your list, or... well, there are many more possibilities, go and give it a try!

I'd like to say that the posibilities are limited only by your imagination, but that's only true if your imagination isn't overly wild. The new system can't do advanced correlation or aggregation, so queries like "give me all visual novels with exactly two female main characters" remain impossible. For that you can always fall back to the SQL query interface.

This system has been developed and tested in the open and I have no doubt that it would've been buggy and shitty as hell if it wasn't for all the testing and feedback provided by rampaa. Still, there's quite likely a few more bugs, performance issues and UI quirks remaining. I'll be addressing these as they come up. The staff search and character search will also be migrated to the new system in the Near Future™ (EDIT: done!). If you had any filters saved as default, these have been automatically converted to the new system. As an additional feature, you can now easily save and load custom queries, too.

May you find the visual novels you were looking for.

sips hot chocolate. it's cooled down as I was writing this ;_;Last modified on 2021-01-18 at 15:42
#2 by ibz
2021-01-10 at 08:55
< report >Thank you, this is amazing.
#3 by klutch
2021-01-10 at 09:05
< report >based
#4 by zik62
2021-01-10 at 09:31
< report >your search system has become complicated and not very intuitive. I'm lost. :(
#5 by naiji
2021-01-10 at 11:59
< report >Thank you, yorhel! I hope you have had great New Year and Christmas
#6 by kiru
2021-01-10 at 12:30
< report >Unfortunately (and to be expected) the automatic settings carried over to the new system broke things. I had chosen to not include "very short" times, or rather all of the others. The new settings made it into separate "has to be" parts for every single time but very short. In other words, there'd never be any result, as nothing can be short, medium, long and so on at once.
Unfortunately that kills my default search now, even though I made a working profile, which I'd assume is that cookies got broken as a result as well. The profile isn't loaded, but the old setting is. I'd assume it would get fixed if I deleted cookies, but I'll leave it like this for now, in case that might be helpful in some way.

I'm sure some people will be a little confused because of this. :)Last modified on 2021-01-10 at 12:31
#7 by zakamutt
2021-01-10 at 12:46
< report >I'd be helped by a character name search. Now you might be thinking, why? Surely character search does that. Well, one thing I sometimes want to do is search by two names to try to find what vn someone's posting screenshots from (I'm too antisocial to actually ask anyone). With AND being an option in the new advanced search, this seems possible without going all the way to SQL as long as you can search by name (I tried, it didn't even give me an error message running my probably wrong query, I quit).Last modified on 2021-01-10 at 12:48
#8 by mutsuki
2021-01-10 at 13:13
< report >I agree with #4. I think it'd be better to have a toggle between the original search and this. Like I could manage to do something with the new search but it seems like a lot more hassle than the old one...
#9 by teardrop
2021-01-10 at 13:37
< report >Branching is good.
Having everything in one line or column and forcing users to read a formatted text file (that's what new system looks like) is NOT good.

If I want to work with tags, I want to work in a separated tab, where I can branch, invert and blacklist tags only. Hear me out, open VLC settings, imagine every tab and every setting had to be added via dropdown menu, now imagine you try to expand it and add more options, that's a disaster, a mile long dropdown list where half of things are boolean and should be incorporated in a simple menu with checkboxes.

Like I said, being able to branch is a good thing, but, realistically, it works for tags and staff only. Two time frames - two different search queries, everything boolean including languages should be a checkbox in a separated tab, and you keep filling these tabs with more boolean things instead of piling up options in dropdown list.

New system also shows when there's an option without explanation, it's hard to use. "Rating" and "Popularity" are just some numbers, how a user is supposed to figure out what they mean, let alone use them for branching as an additional clause. The answer is they have to be in a tab, and potentially there'll be "weight" or "relevance" as well one day, but in that exact tab with a (?) popup to expain what the option does and how math works, if want us to use something so close to syntax, provide a convenient hint.
Btw, actual meaningful filter by rating twice or more (rating above X, branch or/and, rating above X) is pretty much an indication of two different queries, where the second is the lowest common denominator/vastly different thing if the primary search query has failed. Hence, if it's supposed to be used once, it shouldn't be an option to add, just a slider in the main tab alongside time frames.

Search exists to narrow down the scope, we don't need to shoot 4 bottles with 4 bullets and 40 lines long query, it looks like amazing puzzle to solve, but it's FOUR different queries fused together.

P.S.
"Anime" is redundant, "has anime" is superior, one option with a checkbox / inversion will do, I understand you did it for consistency, but that's how problems start. Again, this is terrible for future expansion, it should be in a tab called "Media" or something, and DB eventually should feature at least anime adaptations and drama CDs, maybe even beyond that. As of now, it's already hard enough to deal with Japanese -> romaji conversion, "Anime" already doesn't pass the branching check, I can only think of two different spellings to make use of "Anime" branch, otherwise it's two different searches again.Last modified on 2021-01-10 at 14:14
#10 by freddiex
2021-01-10 at 13:48
< report >Intuitiveness - or lack thereof - aside, how can we change the default search filter? I can't see a replacement function in the new system?
#11 by yorhel
2021-01-10 at 14:08
< report >
I had chosen to not include "very short" times, or rather all of the others
Oops, the old length filters got AND'ed together rahter than OR'ed. You can easily fix that by changing the query and overwriting the old default in the Save/Load button. Fiddling with cookies won't do a thing, I'm afraid, except log you out...
EDIT: Fixed all multiple-length-filter conversions now.

how can we change the default search filter?
Save/Load button -> default.

@teardrop: I agree that the UI isn't the most intuitive and could be improved, but you'd be surprised at the kinds of things people want to query for. There is no such thing as a "filter that is only used at most once", especially not when subqueries are involved.Last modified on 2021-01-10 at 14:31
#12 by weter
2021-01-10 at 15:06
< report >Oh yes finally! Another function that I and many others have mentioned have now been introduced. I love VNDB the best site that is constantly evolving and getting better. Thank you very much for the development of the site

I still hope that someday what I wrote 1.5 years ago here - t12216.1 will be corrected, at least in the section -About Traits-Characters- and it would also be nice to improve the look of the tags as I mentioned in the section -About Tags-

Thanks again for the work on improving the site, great site, great community
#13 by teardrop
2021-01-10 at 15:34
< report >@yorhel
>You'd be surprised at the kinds of things people want to query for.
People want expanded tag system and quick navigation, instead they have to wrestle with basic redundant functions accessible via dropdown. They want to query for specific things that are "A but not exactly B, if no then C with a splash of A, and since we're scraping the barrel X,Y,Z should be voice actors." It's always tags, boring default options like time or language are "set it and forget it," and used to be just fine when they were sitting in a separated tab.

>There is no such thing as a "filter that is only used at most once", especially not when subqueries are involved
There are. In your case it's:
1) Time frame, OR doesn't make sense since it basically turns the whole branch into a different query, AND is redundant since bigger or smaller time frames should be set initially and only once, not as a subquery, the resulting output with two different time frame looks like witches' cauldron. Try to come up with a sane and meaningful output incorporating 3 gaps in time instead of simply covering the bigger time frame.
2) Language - a list of checkboxes, global, you just need to set it once per query.
3) Any media - edit box, an option with two checkboxes: exist, invert (exclude ones with media type available). If you want smart kids to play with this option, you add masks instead of forcing to check every name via subquery.
4) Screenshots - yeah... AND is like, what the heck, if we could set up the level of profanity on screenshots we're looking for, then it would make sense, but "has screenshots" is a global checkbox for the whole query. OR makes little sense, again, no control over what kind of screenshots we're looking for, it's an option that better be clearly displayed in the main tab with a few arguments.

Those are obvious, could be more. The bottom line is you need better containers for many options rather than a dropdown menu, and in many cases being able to include an option in a branch doesn't make any sense, other than entertaining SQL maniacs (they already have API is for that, website could use better tags/staff DB clean up instead)
#14 by donkeyskin
2021-01-10 at 16:55
< report >In the release page, how would I be able to show only results from the releases I own and added in my visual novel list?
Sure, I can choose to filter labels, but the results are including releases I do not own.
#15 by desann
2021-01-10 at 17:08
< report >Is it possible to search for VNs by only parent tag? For example, I want to search for VNs with Japanese Mythology, but I don't want to see VNs with only Youkai Heroine (which is child tag for Japanese Mythology). If I add ¬Youkai Heroine to filters, I will lost VNs that have both Japanese Mythology and Youkai Heroine tags.
#16 by goldgrande
2021-01-10 at 17:18
< report >Awesome!
#17 by NowItsAngeTime
2021-01-10 at 19:49
< report >This is a cool feature!

I look forward to the character advanced search being implemented since some VNs I actually consider if at least 1 character has a combination certain personality traits I like.

The sexual tags being able to be searched in detail is very nice, especially not even hentai sites have advanced tags that detailed. Would be nice to see this in the character advanced search as well.
#18 by acborgia
2021-01-10 at 20:36
< report >This is a really good change and I'm gonna use it a lot, but would it be possible to sort using parameters too?
One problem I was having is trying to find recently translated VNs, so I selected the ones translated last year, but I had no way to know which ones were the most recent since sorting by release date uses the initial release date. Of course it's not a big problem here but when you have hundreds of VNs to sort I think it could be useful
#19 by nagashiedogawa
2021-01-10 at 21:06
< report >I was hyped about this new search system until I tried out what I've been wanting to do for a long time. Tags with the seiyuu. I tried searching Netorare with Himari, and it returned no result. There are definitely VNs with Netorare that has Himari as one of the seiyuu (Kedamono tachi no Sumu Ie de being one of the example) yet nothing's appearing on the search result. Why is this?
#20 by yorhel
2021-01-10 at 21:19
< report >
In the release page, how would I be able to show only results from the releases I own
Aye, I already had a note that such a filter may be useful, will implement later.

Is it possible to search for VNs by only parent tag?
I'll probably be adding that option in the future, yup.

One problem I was having is trying to find recently translated VNs.
For this you should probably use the release search rather than VN search. That allows you to use all the same filters as for VN search, but you can also sort by the release date of the release that matches your filter.

I tried searching Netorare with Himari
Voice actors aren't listed as "staff" under VN entries, so this doesn't work, the correct approach is to go through a character filter. I'll see if it makes sense to fix the staff filter to also match voice actors, as that does sound like sensible behavior.
#21 by basado88
2021-01-10 at 22:02
< report >How do I hide VNs based on a tag?
#22 by acborgia
2021-01-10 at 23:10
< report >Oh I never knew about release search, thanks
#23 by elias67
2021-01-10 at 23:17
< report >Is it possible to search for vns that were first released in English in a given year (eg exists EN complete release date = 2020 and NOT exists EN complete release date < 2020)? I notice you can't invert all fields, and there's no global NOT operator to exclude a parameter.
#24 by fibag
2021-01-11 at 01:47
< report >It's sems that the new system search it's more unfriendly to users and for me not work the story animation or eroge animation filtre I selected example story animation and I press search and nothing happen....
#25 by chronopolize
2021-01-11 at 02:48
< report >Is it possible to have an option to filter rating based on actual average as well? When searching for unpopular but highly rated VN's, the bayseian average kind of muddles things.