Dialog searching utility: useful?

[mzz]

So I'm playing around with a new use for my .dlg parsing code: dumping all the information to a searchable database. This lets me search really easily for things like:

• At what places is Seduction >= 9 useful? (apparently just for seducing Chunk, and 10 is not used at all... similarly intimidate > 8 is useless, but 8 gives you one option I think is funny )
• Is the name "Tourette" actually used in conversation? (no, just as a file name).
• Is it ever useful in dialog to have a shotgun on you (quite the opposite: you cannot say "You're lucky I left my shotgun at home. Goodbye." to vandal if you have a shotgun with you! I bet you didn't know that yet!)

and less easily because I have not come up with a good interface yet:

• How much persuasion do you need to get the sisters to get along? (seems to be 4)
• Anything else that needs to filter on more than one thing at once.

What this does not give you is anything not specified in the conversation files. What it does give you is a very fast (fraction of a second on any system capable of running bloodlines) search engine-style search through the text and a pretty reliable way to search for stat and item checks.

My question is: would anyone but me find this useful? Currently I only have a very rough text-based interface to the db, but I could work on that if others would use it.

Also: what kind of questions do you want it to answer? The database knows many things but they're not all accessible yet unless you write the sql query by hand. For that persuasion-for-tourette example above I manually wrote a query for persuasion checks in tourette.dlg ("all persuasion checks" gives way too much output). It's also easy if you happen to remember some key phrases (a search for "cease quarrel" gives only one result and has the check, because malks say "Become mute both mouths! Cease this quarrel.", but who remembers that...).

(for the technically inclined: the code is written in python and stores the data in an sqlite db, using sqlite's fts3 for fast full text search. The .dlg files contain python expressions which I parse using python's own parser so I can reliably store things like stat checks and SetDisposition calls separately instead of treating it all as generic script. Arguably overkill but I have some ideas that need those checks in a machine-readable format.)

[Dheu]

Searching wise. hmmm..

I am a VTMB developer and when I need to know something about the contents of the dialogs, I use cygwin, go to the dlg directory and type:

grep -Hir <SOME_STRING> .

ie:

grep -Hir "Persuasion 9" .

or

grep -Hir "Tourette" .

If you want it to make something that is a "Must Have" for developers, it will need to make something that does more than I can accomplish with grep.

IE: Make a gui editor that makes them easier to actually edit. (There is one that exists, but it doesn't provide update access to all fields)

or

Adding functionality that goes beyond simple text search. For example, I would love to a have some integrity checking functions. grep is not so good at looking for duplicate lines for example.

{1}{NPC Statement}{}{#}{
{2}{PC Response}{}{10}{Persuasion -3}{
{2}{PC Response}{}{20}{Persuasion 3}{

or... Something that could reveal or at least bring attention to any dialogs that may not have a valid response. ie:

{1}{NPC Statement}{}{#}{
{2}{PC Response}{}{5}{Persuasion -3}{
{3}{PC Response}{}{10}{Intimisation 3}{
{5}{NPC Statement}{}{#}{
{6}{PC Response}{}{10}{G.Show==1}{
{7}{PC Response}{}{20}{G.Show==0}{

What happens if a person comes to line 1 with Persuasion 3? There is no valid response. However, such a function would have to understand the protocol enough to be able to extract dialog blocks (seperated by {#} in the line number area) and actually parse and evaluate all the condition blocks.

Of couse the best thing in the world would be a gui dialog editor that had all those things. But I will take what I can get. =)

Hope this is useful.

Dheu

PS:

As for your tourette question... VTMB makes use of a lot of global variables that get set in diffierent locations. When you seduce Jeanete G.Jeanette_Side gets updated by 1 (updated by Jeanette.dlg) and when you suck up to Therese, G.Therese_Side gets updated by 1 (updated in Therese.dlg). To get Tourette, you must have sucked up to BOTH of the sisters consistently AND have a minimum persuasion.

[mzz]

If you want it to make something that is a "Must Have" for developers, it will need to make something that does more than I can accomplish with grep.

I know about grep Some reasons you might use this thing instead include: it's faster (no big deal, grep is usually fast enough), the output is more readable, it can more conveniently (once I've actually implemented this) navigate through a conversation, it can (once I've actually implemented this, haven't even started on this yet) display the conversation graphically (as a treeview or the like, this will take some experimenting and/or suggestions to be useful).

IE: Make a gui editor that makes them easier to actually edit. (There is one that exists, but it doesn't provide update access to all fields)

I haven't thought about editing at all yet. In its current state the code does not really allow that because it does not preserve line numbers. It could spit out a completely new .dlg file but its contents would have completely different line numbers, breaking python code referring to them as well as confusing other people or tools using the same file (like source control).

Adding functionality that goes beyond simple text search. For example, I would love to a have some integrity checking functions. grep is not so good at looking for duplicate lines for example.

You may have noticed me spewing .dlg improvement suggestions at Wesp (in a few of his patch threads). Those are from reports from mostly the same code Suggestions in this area are appreciated.

{1}{NPC Statement}{}{#}{
{2}{PC Response}{}{10}{Persuasion -3}{
{2}{PC Response}{}{20}{Persuasion 3}{

Not sure what you're after here. The example is not a problem because the "destination" differs, right? Which buggy forms of this pattern do you want me to catch? I've considered checking for cases where it's possible either both or neither of the replies are reachable because the requirements are not met. Do you mean that? Or do you mean cases where the destination (next npc statement) is actually identical, so the pc responses should be merged?

or... Something that could reveal or at least bring attention to any dialogs that may not have a valid response. ie:

{1}{NPC Statement}{}{#}{
{2}{PC Response}{}{5}{Persuasion -3}{
{3}{PC Response}{}{10}{Intimisation 3}{
{5}{NPC Statement}{}{#}{
{6}{PC Response}{}{10}{G.Show==1}{
{7}{PC Response}{}{20}{G.Show==0}{

What happens if a person comes to line 1 with Persuasion 3? There is no valid response. However, such a function would have to understand the protocol enough to be able to extract dialog blocks (seperated by {#} in the line number area) and actually parse and evaluate all the condition blocks.

That's exactly what my code does. It builds a graph (npc lines are nodes, pc lines are edges connecting those). It also parses requirements and does limited (work in progress) parsing of other kinds of script, which I'm hoping to use to do checks for unsatisfiable conditions reliably even in slightly more complicated cases than your persuasion/intimidation example. It's not just a database full of individual lines, it knows which ones are pc and npc and how they are connected.

Of couse the best thing in the world would be a gui dialog editor that had all those things. But I will take what I can get. =)

Hope this is useful.

Definitely useful, I'll see if I can get the sanity-checking stuff in a form others can use too (the output is currently a bit cryptic, I've been translating from its gibberish into something closer to english when reporting possible problems to Wesp). Do not hold your breath for a gui editor, but a (gui or text-based) tool reporting possible problems is doable (mostly already there).

As for your tourette question... VTMB makes use of a lot of global variables that get set in diffierent locations. When you seduce Jeanete G.Jeanette_Side gets updated by 1 (updated by Jeanette.dlg) and when you suck up to Therese, G.Therese_Side gets updated by 1 (updated in Therese.dlg). To get Tourette, you must have sucked up to BOTH of the sisters consistently AND have a minimum persuasion.

I know, although you're right I should have mentioned that too. Figuring out exactly what the requirements are here is a bit tricky and I can't do it completely automated yet. It does tell you Therese_Seduce and Jeanette_Seduce each need to be high enough, and finding all spots where those are increased (in dlg code! changes through level script aren't caught, but I don't think there are any in this case) is easy too, but combining all that to "you must have these stats and followed one of the following possible paths in conversations" is a bit harder...

[Dheu]

{1}{NPC Statement}{}{#}{
{2}{PC Response}{}{10}{Persuasion -3}{
{2}{PC Response}{}{20}{Persuasion 3}{

There are 2 lines numbers with the line id "{2}". The dialog engine will ignore one of them randomly. These are normally fat finger mistakes since most editing is currently copy/paste using notepad.

If you are building in Python, I am pretty certain there are gui libraries available.

[mzz]

{1}{NPC Statement}{}{#}{
There are 2 lines numbers with the line id "{2}".

Duh, should have seen that. I catch those already (the parser complains if that number does not match the actual line number, which obviously also catches duplicates like this). Never checked if that number actually matters to the game though (it seems a bit redundant since it always matches the line number, so I was wondering if it was only there for editing in apps that do not show the line number or something like that).

If you are building in Python, I am pretty certain there are gui libraries available.

I know and have used them, just haven't added a gui to this thing yet (because I do not need one myself and I'm the only user so far). For a .dlg sanitychecker would you prefer a terminal app (with output similar to the warnings/errors from a compiler) or some kind of simple gui?

[Dheu]

Be aware that line numbers do not have to match. In fact, I often leave ranges out on purpose

{100}{}{}{}
{1000}{}{}{}

Second, be aware that a duplication is sometimes intended. There are dialog engine bugs that you can take advantage of by giving 2 lines the same number ofnpurpose. (I mention it in the VTMB Mod Developers Guide). So you dont want the tool to auto-correct, only hi-light.

For integrity checking... terminal is fine until a gui is made. But obvisouly if you have a gui for editing, it makes sense to put a button in that performs the function automatically.

Other things of note: When editing an existing dialog, you dont want to change the line numbers, even to renumber, or the audio files will no longer match up with lines. However, an editor should give authors the freedom to change the number or add new lines anywhere in the file. That is my biggest gripe with the current dialog editor. The file has to already exist and it will only let you edit the PC responses. You can't edit the condition blocks, renumber or add new lines that don't already exist.