Advanced search: (too) early testing & feedback

Posted in

#26 by Yorhel
2020-11-30 at 14:39
< report >
I'm getting an "Internal Server Error" from the following query: link
Fixed that performance issue.

Also did some tests with making all tags searchable, looks like it's perfectly doable if I change some indices. I'll do that after retiring the old search functionality though, the changes may impact performance of other parts on the site and I'd rather not waste time debugging and fixing old code that'll be deleted soon enough.

Do you plan to implement parent-only search? (t12970.11)
...if enough people want it, perhaps. Problem is that it would require maintaining an additional cache in the database, which I'm not sure is worth the hassle for a rarely used feature.
And I just realized that this same cache can be used to shave ~15ms off of popular VN pages as well. Not terribly significant, but that makes it an easier sell. :)
#27 by rampaa
2020-12-03 at 02:42
< report >
Fixed that performance issue.

did some tests with making all tags searchable, looks like it's perfectly doable if I change some indices

that makes it an easier sell

A bunch of good news eh? Great! And I see that you've added a bunch of other filters recently, thanks for your hard work again. It seems like after you add one more filter to release filters (e.g. Publisher) I will have to scroll down to see the all filters available. So here's a suggestion: How about displaying filters like link? That way you could list 2x filters within the same area and I wouldn't need to scroll down.

Edit:
How to search for VN with 2-3 same traits common in all or few characters
This is actually partially possible with the current filters. Take link for example. This returns VNs where all female main characters are blonds with big breasts.Last modified on 2020-12-03 at 03:00
#28 by rampaa
2020-12-04 at 03:29
< report >Mode selection of original language filter is acting odd. In link there's no "Mode:all". link, on the other hand, has "Mode:all" but lacks "Mode:any". Note that this doesn't happen with the language filter's mode selection.

EDIT: It seems like the same thing happens with language filter as well. See link and linkLast modified on 2020-12-04 at 03:40
#29 by rampaa
2020-12-06 at 00:37
< report >FYI: link didn't solve the problem entirely. The original language filter in link doesn't have "Mode:all" and in link it doesn't have "Mode:any".
#30 by Yorhel
2020-12-06 at 07:10
< report >
The original language filter in link doesn't have "Mode:all" and in link it doesn't have "Mode:any".
As for the first link: the "original language" should eventually become a regular single-value field for VNs (as discussed in t12465), so "Mode:all" wouldn't make sense. As for the second link: can you still recreate that from scratch with the current UI? The thing about the URL format is that it's far more powerful than the UI is capable of handling, so "invalid" URLs may result in a few minor display/behavior glitches.
#31 by rampaa
2020-12-06 at 07:17
< report >
As for the second link: can you still recreate that from scratch with the current UI
Yes. link will transform into link when you hit the search button.
#32 by Yorhel
2020-12-06 at 07:37
< report >Heh nice, the power of flexibility. :D
#33 by Jazz957
2020-12-06 at 21:24
< report >Are you planning on adding filters for staff/VAs? So you can search for all VNs from developer X with characters voiced by VA Y for example.

Edit: Hmm. I guess you can kind of already do that with the current search.Last modified on 2020-12-06 at 21:28
#34 by rampaa
2020-12-09 at 10:33
< report >Long tag/trait names overflow link link
#35 by Yorhel
2020-12-09 at 14:54
< report >@34: Fixed.

I've also added filters for VN labels, but it works slightly differently from the old VN list filters: Your user id is included in the URL, so if you pass the URL around people will see the VNs on your list rather than theirs - provided that the labels you filtered on haven't been flagged as private. Unfortunately, the UI doesn't recognize such queries (yet), so they'll also see a (or several) ugly "Unrecognized query" boxes.
It's also possible to manually construct URLs that work the same for everyone, i.e. that do not contain your user id, but you can't create these with the UI (yet?) and people who aren't logged in will get a server error when trying to open such a URL. I'll need to work on improved handling of validation errors, in any case. Server errors aren't very useful.
#36 by rampaa
2020-12-09 at 17:37
< report >I think the last update broke the "Role" filter. See the first two queries from t15139.1 for example. Even if I try to re-add the role filter manually, I still get an "Extraterrestrial Server Error". Did aliens finally decide to take over VNDB?Last modified on 2020-12-09 at 17:38
#37 by Yorhel
2020-12-09 at 17:42
< report >My bad, I shouldn't be doing deploys while I have uncommitted changes. Fixed if you do a full refresh.

EDIT: And yeah, there are no bugs, only alien interference.Last modified on 2020-12-09 at 17:45
#38 by rampaa
2020-12-09 at 17:54
< report >Is link how you aliens intended to display unknown platform selection?
#39 by Yorhel
2020-12-09 at 17:57
< report >Aaaaaaah crap I made a note to change that but the aliens made me forget!
#40 by rampaa
2020-12-11 at 04:18
< report >Long trait names still overflow before they are selected: link
#41 by rampaa
2020-12-13 at 14:06
< report >Can label dropdown be columnized? Because of the same rationale as t14995.27.

It's probably me being a dummy but when I saw "ID" on staff filter, my initial reaction was "Do I need to enter the VNDB ID of the staff o.O" (ie. "2" for Nasu Kinoko). So maybe consider renaming it to "Name"?

If staff is going to have its own column, is there any reason to have it under VN column as well? As far as I can tell, selecting VN->Staff does the exact same thing as selecting Staff->ID, so maybe consider removing it from the VN dropdown? Alternatively, and I would actually favor this over the aforementioned solution, you could remove the Staff column and we could add the ID/Language/Gender filter for staff after adding the staff filter. So selecting VN->Staff would yield link and we would add the ID/Language/Gender filter by clicking the + button after that. This is basically how we are specifying seiyuu's gender at the moment anyway.

I think something like "Japanese VNs that have an English person as a staff member" would not be a particularly useful/interesting query, but something like "Japanese VNs that have an English person as a scenarist" might be. So maybe consider adding a role filter for staff?

Now that you have added seiyuu and staff filters, I believe you have adapted all the old filters to the new advanced search. Congrats!
#42 by Yorhel
2020-12-14 at 15:16
< report >
Can label dropdown be columnized?
Eh, maybe. Problem is that some people have pretty long names for labels, so that's not always going to work well.

"ID" on staff filter
Hah. I initially did call it "Name", but that implies it is doing a search on the specific alias you've selected rather than on all aliases of the selected staff ID. I agree that "ID" isn't the best, but it does need to be made clear what this filter does. (And you can search for ID if you use the full format: s2)

If staff is going to have its own column, is there any reason to have it under VN column as well?
Ease of use / convenience, I guess? I don't think many people will be using other staff filters than direct ID lookup, so it makes sense to have a shortcut for that. Using your approach (which is what I did with release/character subqueries before adding "tabs" to the [+] dropdown) requires more clicks and isn't all that intuitive, IMO.

So maybe consider adding a role filter for staff?
Aye, we'll need that anyway for feature-parity with the staff filters (then we'll have fully adapted the old filters), but I need to work out the exact meaning when used in a seiyuu filter, i.e. does "has a seiyuu that is also a scenarist" imply that it is the scenarist for the VN or that the seiyuu has performed that role for any VN? The former should obviously be the meaning when used as a staff filter under VNs, the latter is what'll be used when searching for staff directly.Last modified on 2020-12-14 at 15:20
#43 by rampaa
2020-12-14 at 15:52
< report >
requires more clicks and isn't all that intuitive
Isn't that exactly how we are adding a seiyuu filter though? In order to search for a character voiced by a female seiyuu, we need to add Character->Seiyuu and then click on the [+] to add the gender filter. If we need to intuitively understand that for seiyuu filter, then we can understand it for staff filter as well. Having staff as a tab and under VN is more confusing than helpful IMO.

On a somewhat related note, in link "Staff" should be renamed as Seiyuu, if it's always going to result in something like "Seiyuu->Filter" (like link) to avoid confusion.

then we'll have fully adapted the old filters
On my defense, we couldn't use staff role as a part of the VN search in the old system, so technically you have adapted all the old filters for VN search.

Edit: Should we report queries that result in "Extraterrestrial Server Error" (presumably due to time-out), or do you check the logs periodically? I've managed to get an "Extraterrestrial Server Error" with the staff filter yesterday but couldn't be sure whether or not it would be pointless to report it.Last modified on 2020-12-14 at 16:02
#44 by Yorhel
2020-12-14 at 16:16
< report >
Isn't that exactly how we are adding a seiyuu filter though?
Unfortunately, yes... because the [+] dropdown can only handle one level of nesting while adding a seiyuu filter to VNs requires two (-> Char -> Seiyuu). Hmmm, maybe I can add support for another level with an additional row of tabs.

Should we report queries that result in "Extraterrestrial Server Error" (presumably due to time-out), or do you check the logs periodically?
I do. I was just debugging one of your timeouts. The seiyuu filter has a serious performance problem - what it does is search for a character that has a matching seiyuu assigned to the VN, as you'd expect. But if I relax the requirement to searching for a character that has a matching seiyuu in any VN, the query goes from 10+ seconds (=error) to 70ms. The result will be the same except when a character is linked to multiple VNs with different seiyuu, in that case the faster query may result in a few extra VNs. Ugh. Might have to sacrifice correctness in the name of performance. :(Last modified on 2020-12-14 at 16:19
#45 by rampaa
2020-12-17 at 12:41
< report >I am not sure if the following are already planned, but I will list them just in case:
-"Primary language" and "Type" filters for developers/producers
-"Unknown" option for developer/producer/engine filters (just like how we can search for unknown resolution and/or age rating)
-VN search by anime relation (You have already said you will implement it but I thought you would implement "Has anime" functionality as a part of this filter. Since it didn't happen that way I thought maybe you have forgotten about it.)Last modified on 2020-12-17 at 12:41
#46 by Yorhel
2020-12-17 at 14:04
< report >
-"Primary language" and "Type" filters for developers/producers
To be used only on producer search or also as subquery in the other listings - like how the staff/seiyuu filters work? I did consider reworking the producer filter (which I wrote early on) into the staff-like subquery approach, but it complicates the UI a little bit and I wasn't sure anyone would care.

-"Unknown" option for developer/producer/engine filters (just like how we can search for unknown resolution and/or age rating)
Nothing planned, but maybe. Handling "unknown" filters is pretty annoying, so I'll be a happier person if I can get away with not implementing them. :)

-VN search by anime relation
I do have this on my to-do list filed under "not critical for 1.0", main reason I've not implemented this yet is because I have doubts about its usefulness. I'd expect this would mainly be used to finds VNs in a particular franchise or setting, but "franchise" and "anime" are very different things. I believe tags are better suited for this, but then you get into discussions about which franchises are worth creating tags for...
#47 by rampaa
2020-12-17 at 15:23
< report >
To be used only on producer search or also as subquery in the other listings - like how the staff/seiyuu filters work?
The latter, like how seiyuu filter works.

I believe tags are better suited for this
I think tags are ill-suited for the task for the most part. Mods -rightfully- won't approve dozens of tags for different anime franchises so we won't be able to search for them through the UI in combination to other filters. I agree that this is not a critical filter but it's surely something nice to have.Last modified on 2020-12-17 at 15:24
#48 by otakudoug
2020-12-22 at 03:49
< report >I understand there is already a way to blacklist visual novels, but is it possible to add a whitelist where whitelisted visual novels always show up in searches. for example a visual novel with a tag that is being excluded will still show up in searches. Also to make it so that blacklisted visual novels don't show up even if searching in a tag that it contains.
#49 by Yorhel
2020-12-23 at 10:45
< report >@otakudoug: Yes, you can construct your query like this to get what you want.
#50 by rampaa
2020-12-23 at 19:59
< report >Trait IDs are shown with the wrong prefix (g, instead of i) on trait filter: link

On that note, can those IDs be made clickable on filters too?Last modified on 2020-12-23 at 20:00