Announcement

Collapse
No announcement yet.

My "everything" app

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    I spun a new math expression parser off of my generic parser and it is elite. It supports all possible math constants and functions and I was even able to include modulus support. ​​​It's barely 80 lines after removing comments and unnecessary whitespace (other than new line).

    My method this time was way smarter. First I removed all whitespace. Then I found every + and - symbol that was a polarity operator and replaced it with a POS or NEG token. Finally, I sequesters all parenthetical groups. Doing all that turns the expression into a script. As an example

    -ceil(1.5)+-6

    would become

    NEG:ceil$ROUND1+NEG:6

    If there is any confusion regarding $ROUND1... Lib[$ROUND1] would equal 1.5 from the example above but it can be any valid math expression.

    From there it's simple

    1) split expression on operators [+-/*%] to array
    2) iterate array splitting each index on : and $
    3) determine polarity or null
    4) determine lib name or null
    5) if lib name determine func or null
    6) if lib name compile(lib[name])
    7) if lib name... if func apply func
    8] if not lib name apply numeric or constant
    9) regardless of lib name apply polarity (if any)
    10) loop operator stack inside operator priority loop applying match operators to left side value stack index.
    11) slice last used operator from operator stack
    12) slice right side value from value stack

    The end result is the first (and now only) value of the value stack is the numeric result of the expression...cake.

    I do 99% of this in one procedural loop.

    Next step is to tie the math parser and object parser together so objects can contain math expressions. I will also need to upgrade my reference system a little so references can be injected into expressions.
    Last edited by MadGypsy; 10-04-2017, 03:53 PM.
    http://www.nextgenquake.com

    Comment


    • #32
      I completed both of my parsers, tied them together and upgraded the reference system.

      The right side script is my test object. The left side script is my main document class and it's really only showing you what I am tracing. The bottom output is the result of the trace statements. As you can see, the expression is having references properly replaced otherwise the "eval" value would not match the "math" one. Aside from upgrading the reference system I also changed the reference delimiter. It used to be %some.reference% now it's simply @some.reference. There is no need for a trailing delimiter because a reference will always end up split at it's end. When a reference is used as a name it will always be the entire name and when it is used in math it will either be followed by some split character or nothing at all. Either way, the end is always the end and does not need to be delimited.




      @edit

      I also had to make a correction. I stated that my math parser supports all math functions and constants. NOW that is a true statement. I overlooked the fact that 4 math functions (pow, atan2, max, min) expect 2 or more arguments...oops. It works fine now as you can see from the trace below



      I wanted to show y'all something else. The below image is my old and original delimiter parser for math expressions.



      The next image is of my current generic delimiter parser.



      What I find interesting about this is, I did NOT take my old script and upgrade it. I stumbled across the old script when I went to rescript my math parser. I wrote the new script BEFORE refinding the old, with the intent that it would be solely for object parsing. Once I completed it, I realized it was too universal to be locked to one reason which is what spurred me to rewrite the math parser (hence finding the old script). So, in short, I essentially wrote a more-or-less identical script by chance. The only reason the new script is so much more "fancy" than the old is because the old only had to worry about parenthesis, the new can parse any parenthetical group BUT these scripts are doing identical things... finding inner and outer bounds of a delimited substring and returning those bounds. If you can see beyond the massive comments and how I made certain things more generic you will notice that these scripts are practically identical, period. Take out all the quote exceptions and my "identical" statement becomes complete fact.

      I'm not sure how to look at this "by chance" identicalness. On one hand, I could believe that things I've decided are what I've decided and I will decide them again even if I don't remember the initial decision... maybe meaning that I sort of unconsciously store well working methods/solutions. OR it's a simple case of the inability to teach and old dog new tricks. Either way, this stuff is working VERY well and I doubt I will ever bother to create any other way to do it. Holy Shit! I am satisfied. Pbbbbt.... I didn't think such a thing was possible. Maybe my skills are starting to catch up to my generally unrealistic expectation of perfection.

      I have officially created an object notation and resulting Object that smokes json in almost every conceivable way. It also consumes JSON cause there is no reason why my parser can't parse JSON. JSON is just less possibilities than my object notation but all of those possibilities exist in my parser. You do not have to quote "name":value pairs in my object notation but, you could and it would break nothing at all. It would just token the name and then when that part of the script was hit it would acquire the name from the token value in the lib. Basically JSONs stupid quote names just make my parser perform an extra operation to accommodate.

      EDIT: I was right and wrong. My parser will not parse JSON out of the box. I had to write 2 exceptions to get it to parse json but then I decided fuck json and took the exceptions out. If JSON could have been put cleanly into my system I would have kept it but, with lines like:

      if (src first character = { and src last character = }) strip first and last character

      I just can't put shit like that in my parser. too junky but necessary in order to strip jsons stupid "everything" wrapper so, fuck json. Too junky to be supported.
      Last edited by MadGypsy; 10-06-2017, 12:30 PM.
      http://www.nextgenquake.com

      Comment


      • #33


        I added some precompile stuff. I want to also add precompile conditionals just to see if I can. I do not really need that though.

        I had the worst kind of error you can have earlier. The app just stopped. No errors, no eventual "infinite loop" warnings, nothing. Tracking shit like that down is always a nightmare. I actually wound up fixing it by making more mistakes trying to fix it. I don't remember exactly what I did but it made me look at a line and I realized I was over-processing a value which led to a null value that was overwriting everything, something, something...blah blah blah. It made me realize something else though...my parser is running out of room. Before I started dumping compatible ideas in, there was a lot of room to "wiggle". Now, any immediate thought/idea I have is probably not considering something and may require surgery to include. There will be no surgery. I think I am at the end of the road syntactically. However, the precompile syntax leaves me some room. My delimiters for the current possibilities are:

        #import ...[\r;,]
        #package ...[\r;,]

        that being said, I have plenty of room for at least
        #if
        #elseif
        #else
        #end

        I also have room to include one more piece of syntax but, it's for post processing. I already have an Object "merger" where I can make some default Object and then use underscore syntax on a new Object that only needs to overwrite some of the values (or all of them)

        To visualize this,... In my Image is GraphicObject and within Button is main_GraphicObject. If these objects were merged main_GraphicObject (in this case) would contain all of it's properties plus all of GraphicObject properties. If the two objects being merged have set properties of the same name the target object property (ie main_GraphicObject) takes precedence. I guess a simple way to say it would be - the target object inherits any value which it does not possess. However, this is not so easily applied to properties which are objects. In the case of both objects possessing the same child object, the child objects get sent to merge where the same rules as the original merge apply.

        This system is necessary because numerous objects in my system will reuse the shit out of some values. If I didn't have some kind of inheritance system my script would be very bloated with a bunch of copy/paste repeating code. Once I get my merger finished I'll show you the difference between what I have to write and what I WOULD have to write if there was no inheritance system. The difference is substantial.

        I also want to make it possible to ghost imports so they do not get compiled into the final byte object. This is mostly because the final object should have all references applied and mostly my imports are just a bunch of constants so, compiling the imports or at least not making it possible to NOT compile the imports would lead to a whole lot of useless fluff in an object.

        Final Thought (regarding running out of room in my parser):
        I read something a long time ago regarding good OOP that alleged programmers are commonly trying to "build their way out of a box" when they should "build their way into a box". Honestly, I didn't really make the connection then and I don't really make it now, either. But, I think my parser may be an example of building into a box. I couldn't draw a flow chart on why but, other than the couple/few things above I have basically run out of places to go. I'm assuming I filled the box, which in my line of thinking means I had to be building "into" the box because you can never build your way "out of the box". The box can always be made taller. It cannot be made deeper if you start by defining the top.

        hmmm ~ start by defining the top... maybe I do understand the box and I think I also got the thing I read backwards cause, building into a box means you can't fucking go anywhere at some point.
        Last edited by MadGypsy; 10-08-2017, 10:47 PM.
        http://www.nextgenquake.com

        Comment


        • #34
          I got rid of #import (because it is not an accurate description) and replaced it with #include and #use. If a script is included it sticks around forever. If a script is used it is deleted as soon as the script object has been compiled. Use is basically intended to allow large lists of constants to be available for reference compilation. At this point, I am loops, conditionals and functions away from this being a language. I have no intention of going that far though. I have already gone too far to some degree. I mostly just want to use external objects to act as sort of a database of properties for various classes. A number of the things I have added to this will definitely make it a lot easier to concoct my objects but, overall none of it is necessary. I'm just having fun so I keep dumping my ideas in. Plus, with every addition I'm shoving JSON face first in the mud as little better than an xml resyntax.
          http://www.nextgenquake.com

          Comment


          • #35
            Random observations


            Fun fact: I left quotes in my objects for legacy and convention reasons. I could completely remove the need to ever use a quote as a delimiter, without substituting it for some other delimiter. It is really simple why this is possible. There are only 2 nondelimited values in an object...Numeric and Boolean. A nondelimited non(numeric|mathematical) string will not parse to a number, nor would (should) it exactly equal either "true" or "false" so, if it isn't a number and it isn't a boolean, it must be a string. Quotes are unnecessary from an object parse perspective.
            Last edited by MadGypsy; 10-09-2017, 11:31 AM.
            http://www.nextgenquake.com

            Comment


            • #36


              All 4 of those scripts get compiled into one object. All references are then applied. Lastly, merge is run. The final Object is this:

              note: too big for a screenshot but this is a copy paste straight from my output panel
              Code:
              Button:{
                  classname:"ui::Button",
                  main<Object>:[
                      {
                          classname:"ui.core::GraphicObject",
                          shape:{
                              type:"rectangle"
                          },
                          fill:{
                              type:"flat",
                              color:0x555555
                          },
                          line:{
                              thickness:0x1,
                              type:"flat",
                              color:0x111111
                          }
                      },
                      {
                          classname:"ui.core::GraphicObject",
                          shape:{
                              type:"rectangle"
                          },
                          fill:{
                              type:"flat",
                              color:0x777777
                          },
                          line:{
                              thickness:0x1,
                              type:"flat",
                              color:0x111111
                          }
                      },
                      {
                          classname:"ui.core::GraphicObject",
                          shape:{
                              type:"rectangle"
                          },
                          fill:{
                              type:"flat",
                              color:0xdddddd
                          },
                          line:{
                              thickness:0x1,
                              type:"flat",
                              color:0x111111
                          }
                      }
                  ],
                  label:{
                      formats<Object>:[
                          {
                              classname:"ui.core::FormatObject",
                              size:0xb,
                              color:0x777777,
                              font:"sitka"
                          },
                          {
                              classname:"ui.core::FormatObject",
                              font:"sitka",
                              color:0xdddddd,
                              size:0xb
                          },
                          {
                              classname:"ui.core::FormatObject",
                              font:"sitka",
                              color:0x555555,
                              size:0xb
                          }
                      ],
                      embedFonts:true,
                      gridFitType:"pixel",
                      selectable:false,
                      multiline:false,
                      classname:"ui::Label",
                      sharpness:0x190,
                      background:false,
                      autoSize:"left",
                      antiAliasType:"advanced",
                      type:"dynamic"
                  }
              }
              As you can see, none of the #use scripts were left behind. No reference was unassigned. Merge is so aggressive it will merge someVec<Object>[0] against whatever it was told to and then merge someVec[++n] against someVec[0]. This may seem like a lot for 1 Button but, that's because it's not for one Button. All UI elements will inherit from #use Objects and even other UI elements.

              EDIT: Oh, and now all non-Float values are read back as hex. In some cases this is a little weird and I might find a way to make it more specific but, as it stands, dealing with hex numbers where they may not belong is easier than reading colors as base 10.
              Last edited by MadGypsy; 10-09-2017, 06:59 PM.
              http://www.nextgenquake.com

              Comment


              • #37
                I added a bunch of stuff. #defines allow you to define vars that can be used in conditions for precompile. #config vars can also be used in conditions but are mostly to set compiler conditions. There are currently only 2 #config vars (shown in image). I made it possible for #use and #include to contain comma separated values. I also created a 3rd parser specifically for conditions (exampled below)



                on the right I have conditions regarding the value of a test var. On the left I am tracing the results of that var and in the output panel is the result. According to the conditions on the right the value is correct. This means everything is working. Unfortunately, my conditionals are rudimentary. You cannot nest conditionals and it only currently supports these operations..."==", "!=", "&&", "||", ">=", "<=", "<", ">". It also only supports Numeric, Boolean or String types as comparison values. You also cannot nest a #define in a conditional. Well,...you can but it will not wait for the conditional stage to become defined so, the condition would be moot. It is also NOT possible to use a math expression in a condition. However, it IS possible to nest as many parenthetical groups as you like in a condition. I'll make it all better. I've never tried to "invent" conditions before. There is a lot to consider and I am probably halfway through the list.

                EDIT:


                added ! (not) and <> (not equal) support. ! exampled in picture. I'm pretty sure that covers all logical operators.
                Last edited by MadGypsy; 10-10-2017, 06:00 PM.
                http://www.nextgenquake.com

                Comment


                • #38


                  Added loops to the precompiler. They are a bit specialized. The data area gets parsed to an object and is used as replacement values for copies of the template object. You CAN nest conditional statements in the while as long as the conditionals aren't nested in a conditional.

                  Current state of my parser/compiler
                  allowed types: int, uint, hex, hash, Number, Boolean, String, Object, Dictionary, Array, Vector<ANY>, Math Expression, @Reference
                  compile features: define, config, conditions, loops, use, include, template
                  save as: .von(text), .vo(byte object)
                  read: toObject, backToString, fromStringFile, fromByteFile
                  Last edited by MadGypsy; 10-10-2017, 08:33 PM.
                  http://www.nextgenquake.com

                  Comment


                  • #39


                    here is an example of utilizing a conditional within a loop to change my labels to Spanish dependent on a define.

                    And here is an even better way to do it in this context. Instead of setting SPANISH to true|false. I set LANGUAGE to "Spanish". The conditions in this example represent a situation where you would have many languages and "else if" your way through them all.


                    lol, the below works as well. In this case I stuck the conditional in the guts of my Vector. That brings me to another point. As you can see in the image below, doing it this way creates a lot of whitespace. My stuff about 98% gives 0 fucks about whitespace. I could add 50 new lines right in the middle of the guts of anything but a string and it won't register. The same goes for comments. I have tests with comments plopped in a vector tag (ex: <Ob /*this is a comment*/ ject>) and this wont break my script either. My belief is that as close to nothing as possible should break the script, even weird shit like comments in the middle of tags. I have a small list of things to bang out which will make it even more bullet-proof. However, as it stands now you have to make a pretty critical mistake to break anything. For instance, forget a comma in a name:value pair and you will definitely have a problem. Also, missing a necessary comma or open/close parent delimiter is not going to end well. Other than that, (which would break any language) it's pretty hard to break one of my objects.

                    Last edited by MadGypsy; 10-10-2017, 09:34 PM.
                    http://www.nextgenquake.com

                    Comment


                    • #40
                      Thought I just had: could you link into an external dictionary and parse in translated words from English in lieu of manually coding strings in different languages? Maybe not worth lugging around all that extra info though. But that would be cool.
                      'Replacement Player Models' Project

                      Comment


                      • #41
                        Dutch - I could totally do that but, I have already gone waaaaay too far. I just wanted to externalize the objects that I was hard-coding into my script. The thing that kept me going on this was I broke new barriers on parsing objects and created a system that is superior to any other string parser I have ever made. If you saw/understood my system and then looked at my If and While compilers you would quickly realize that making new features is absolute cake. This is how I banged out 2 parsers yesterday in about 15 minutes a piece. I guess what I'm trying to say is that these new features are little more than playing with my system and illustrating how powerful the core of it is. I need to get back to where I was before this parser but utilizing the results of my parser. So, this parser is pretty much a wrap. I might even suppress if and while because I currently have no real use for it.

                        It will probably take me 2 days to rewire my app to use external objects. I'm sure I could actually do it in an hour but there will be more to this than changing some lines from "look here" to "look there" and I don't feel like find and replacing all over the place so, I'm just gonna rewrite the shit. It's like 11 classes and maybe 2 or 3 of those are reusable as is. I'd also rather rewrite than test...fix error list...test....fix error list...test.... until I have addressed everything. This also gives me a chance to make everything better. EVERY time I rewrite my scripts get tighter. There has NEVER been a rewrite that compared to the previous write was a waste of time. In many cases my rewrites make the previous look like a POC.
                        http://www.nextgenquake.com

                        Comment


                        • #42
                          Gotchya. As far as rewrites, I know what you mean. There's nothing like taking functioning code and condensing it down and/or making it faster. I've been trying to do that in my little qc project but the problem is id coded in a somewhat sloppy way: I have to balance between trimmed code, or cross-file dependency. It's good that you're structuring this in a way that seems to avoid both (lengthy bullshit code and layers of dependency).

                          I know that's apples and oranges, but IMO good code is good code. Foundations remain unchanged.
                          'Replacement Player Models' Project

                          Comment


                          • #43
                            Dutch ~ My old string parsers tried too hard. Every possible combination of delimiter situations was defined. For instance, from a close delimiter perspective, object in an object }} , object in an array }]... Etc. This was shit. It worked but, it was too specific. Splitting code on delimiters to create a stack is also shit (until stuff gets segregated). My new method is far more simple. Remove all quotes and comments... then go through the script "blinking" out entire parents (including nested). Once the main source has been reformatted to a string full of meaningful tokens it is simple to send the library value of the token back through the same parser. In other words instead of creating some ridiculous stack of mess right off, I categorize each piece of the script and then address each category. This is my current base system for all of my string parsers and it works great every time. Whether it be enumerable, iterables, base types (string, bool, etc), math, conditionals, loops... The base system doesn't care. It only cares about 2 things. What are the open and close delimiters? And ... Should I tokenize all nested groups or ignore and only token the parentest? Answer those 2 questions and you will ALWAYS get back what you said. It doesn't matter if the delimiters are { and } or #if and #end if (ie simple or complex).

                            So, yeah, THIS system is rock solid and the only dependencies are the parser base to anything that extends it...EXCEPT, the object parser has a dependency on the class for the final object it creates. I could change this but then the dependency would just be in reverse OR I would have some duplicate code. I don't have any qualms with my parser being dependant on some constants in the class it returns.
                            Last edited by MadGypsy; 10-11-2017, 11:55 AM.
                            http://www.nextgenquake.com

                            Comment


                            • #44
                              Dutch ~ I have to balance between trimmed code, or cross-file dependency.

                              Maybe you aren't aware that there basically isn't any files in QC from a compiler perspective. You are just writing one long-ass script spread over a number of files. This is actually quite obvious if you think about it. Where in any of your QC are you importing/including some other file? If I define something in ex: subs.qc I can make a qc file 50 files down from that in progs.src and immediately use the variable/func/whatever from subs.qc without ever making some reference to subs.qc. There are no cross file dependencies in QC. There are no files in QC...just one long file that you are allowed to break into many. OR you could look at it the other way. No matter what you do the entire thing is one giant dependency. Actually all of Quake is one giant dependency if you think about it. In real life lights do not have ammo just because I do.

                              In lieu of all that, trimmed code, trimmed code, trimmed code. Trim the fuck out of it.

                              Edit: However, in QC things before know nothing of things after. In other words something has to be defined BEFORE it can be used. I'm not sure why that is, even my regex object parser doesn't suffer from this nonsense BUT maybe you can use it to your advantage by making the most "knowledgeable" things the furthest down in your scripts. I'm not positive that will really do anything but, it's the closest you are going to get with stock QC to "limiting dependencies". Honestly, IMO, if you want to write some real code and push your limits as a programmer stop using QC and pick a real language with a real compiler and a more versatile purpose. Real talk, bro stock QC is total bullshit. Spike made some great advancements to the language but it is still kitschy at best. At the very least get spike or somebody to teach you QC classes. I'm too "out of the mix" to do it. It's been too long and was something that I only had a passing interest in. I'm not sure QC classes are going to do anything for dependencies though. You would think it would since now you are using classes but I don't recall having to import/include/using anything and if you aren't specifying stuff like that then it must already be available. If it's already available then everything is available and we are right back to one long script. I'm not implying that you don't have to import/include/etc stuff I'm just saying I don't remember that being a thing.
                              Last edited by MadGypsy; 10-11-2017, 11:05 PM.
                              http://www.nextgenquake.com

                              Comment


                              • #45
                                @cross-file dependency

                                Yeah, I was aware the entire QC codebase is basically one big script. I coined that term up so to speak because I've been breaking apart and re-organizing the shit out of everything, and I have been trying to keep only relevant and related functions within a single file for organizational purposes. I've damn near tripled the 'file' count so far. Short, sweet, and relevant. 1500-line files filled with unrelated stuff bugs the shit outta me. So what I really meant by 'cross-file' was the fact that if you change the rocket touch function in weapons.qc, you change the lavaball touch function for chthon's attack, because it simply calls the rocket touch function. Obviously, the location of the function is irrelevant...I could put the touch function in the same file as the lavaball function and nothing changes (except I'd have to add a function declaration in weapons.qc cause it procedurally gets compiled first by the way I have my progs.src file configured). This is no good from a modder's perspective. So my options are: 1) write an *almost* identical function specifically for lavaballs, 2) add classname exceptions in the touch function, or 3) find out if there's a way to carry variables over a touch function. #4 is the best bet and totally do-able if I declare some additonal .float fields, but after the lesson in lingering references, I'll be looking for a better way. Actually, I just had an idea I'll try this weeked.

                                @learn a real language

                                One of these days. I don't stick with QC just because it's basically all I know and I'm fairly familiar with it (core concepts, anyways). I actually still get a lot of enjoyment out of it.

                                Anyways, I digress. How's the rewrite going?
                                'Replacement Player Models' Project

                                Comment

                                Working...
                                X