«

»

Need Some Hack-tastic Help

Categories:

September 14, 2010

by

So I’ve got a problem. Apparently I am not the only one with this problem, nor is it a new problem. That being the case, I wondered if anyone had a more elegant (aka less hackish) solution.

I’ve got a scheduled agent using LotusScript, specifically using the “ComputeWithForm” method to (hopefully) recalculate computed fields. I could swear that this agent used to work, and now does not. Coincidentally, someone recently pinged me to see if something was up with 8.5.2 because he noticed the same behavior (that is used to recalculate and now does not) once he upgraded. A little bit of digging, and a lot of bothering people (thanks Julian!) discovered this blog post from Duffbert (whom I would have bothered, if he wasn’t on vacation!) showing that this is an iffy proposition and has been since at least 2004!

Two workarounds are stated, the first being ToolsRefreshAllDocs, which unfortunately, does not work for me in this case since I get an error about too many documents.

The second workaround is to have the LotusScript agent perform the calculation itself (rather than refreshing the field calculation on the document).

I think both of these sound pretty hack-tastic.

All the other solutions I can come up with are front-end/UI methods and therefore won’t work in a scheduled agent. I supposed I could write a front-end agent and just wake up every morning at 3AM to run it manually.

Anyone else out there run into this? Have a better solution?

Tags: , ,

  • http://dmcnally.blogspot.com Don McNally

    I have run into this repeatedly over the years. Not sure why it looks like it works sometimes and not others.

    Anyway, what I tend to do is remove the items in LotusScript using NotesDocument.RemoveItem and then run the ComputeWithForm. That has worked because it will compute initial values for the fields that are missing. I haven’t seen the behavior mentioned in the technote in Tom’s post.

    Of course, we don’t have 8.5.2 yet so YMMV.

  • Emily

    Not on 8.5.2 either, but have run into this situation a lot of times and have recently started using solutions which look at the DXL to get the current formula for the field from the design (to avoid hardcoding it in LS) and then using Evaluate to compute it onto the document.

    You can either do this on the fly or set up a scheduled agent to store field formulas in the DB somewhere so you can look them up a bit quicker.

  • http://www.poweroftheschwartz.com Richard Schwartz

    I would argue that the type of self-referential incrementing formulas referenced in Duffbert’s post are a bad idea anyhow, because recalcs of fields can and do happen at times that developers don’t anticipate. So I would not call ComputeWithForm “iffy” on that basis.

    However, ComputeWithForm can be quite expensive in large complex applications. An agent, whether formula or LotusScript, that recaclulates only the fields that need to be recalculated, is my usual preference.

    All that said, I have no idea why ComputeWithForm is not working for you in 8.5.2. What happens? Anything? Nothing? Are you calling it with the 2nd argument set to false?

  • http://jonvon.net John Vaughan

    What Rich said: “An agent, whether formula or LotusScript, that recaclulates only the fields that need to be recalculated, is my usual preference.”

    I prefer doing it this way (computewithform is def expensive in complex apps) because I know exactly what is going to happen and I can do my own defensive coding and error trapping. A lot of increased certainty imo.

  • Richard Shergold

    Richard, it was me that Pinged Kathy with a similar issue. I found that as soon as my customer upgraded to 8.5.2 that the documents processed by my agent DID NOT contain the computed items. Previous to the upgrade, the items were there. These items were created using ComputeWithForm and the fields are Computed fields on the form. So I can only conclude that the behaviour of the ComputeWithForm method has changed with 8.5.2.

  • http://www.runningnotes.net Kathy Brown

    Here is what I don’t get. Open the form in the UI, refresh or save and everything is fine. No errors on validation. Yet, the very same document will fail ComputeWithForm. Why?

    And define “self-referential” in terms of what fails in ComputeWithForm. Is a formula that uses @ThisValue self-referential? If so, that’s a pretty big failure in my opinion.

    I am not using 8.5.2, by the way. I simply mentioned it as additional color.

  • http://www.runningnotes.net Kathy Brown

    @Richard and @Jonvon – “recalculate just the field” – isn’t that the second workaround? Or is there some better way I’m not thinking of?

  • http://www.poweroftheschwartz.com Richard Schwartz

    @6 Re self-referential, I was referring to the issue mentioned in Duffbert’s post, where the formula in field Count was simply “Count+1″. And yes, “@ThisValue+1″ would have the same issue. And what I’m saying is that both of these formulas are bad ideas in computed fields, irrespective of whether or not there’s a computeWithForm going on. A formula for fieldA that uses @ThisValue as the key for a lookup whose value replaces the value of fieldA would be a similarly bad self-referntial formula.

    @7 Yes, recalculating the field is the “second workaround” referred to in Duffbert’s post.

    @5 & @6 It sure does sound like you’ve found a bug in computeWithForm in 8.5.2. Have you tried to reproduce this in a simple case? There are some other possible explanations, though they do seem a bit far-fetched. (As in there’s always been a problem in one of your formulas, and this is the first version of Domino that actually detects it, and it doesn’t show in the UI because the formula actually does work in that context.)

  • http://nathan.lotus911.com Nathan T. Freeman

    “So I can only conclude that the behaviour of the ComputeWithForm method has changed with 8.5.2.”

    Actually, you can conclude that something with the evaluation of @formula has changed. It might have nothing at all to do with .ComputeWithForm as such, but a formula that used to evaluate to something, and now throws an error instead. (This could be intentional or it could be a regression.)

    Bear in mind that the new error condition could be a side-effect of some other change, such as a default ACL or a change to a built-in template such as the NAB.

    It is vastly more likely that your FORM is broken than the LS method is broken. Even if you think it worked before (look, I can open and refresh the form in the UI) doesn’t mean it works in whatever run context you might be executing the .computeWithForm. For instance, you might be performing a @DbLookup where you have a server name specified. Even if it’s the same server that’s executing the .computeWithForm, this can be considered a remote lookup, and therefore fails when the function is evaluated.

    In general, the use of .computeWithForm is just a horrible, horrible idea because it provides you with so little information about what went wrong. Even if you intercept the error, all you know is that SOMETHING threw an error, but not even what field or the resulting message.

    If you have complex data calculations, better to refactor them in Lotusscript and/or Java, and then have BOTH your UI form and your backend agents calling THOSE methods. You users will also thank you because they’re applications typically get MUCH faster.

  • http://www.poweroftheschwartz.com Richard Schwartz

    Upon reading Nathan’s response, I take back what I said about other explanations being far-fetched. They are, in fact, quite near-fetched.Emoticon

  • Richard Shergold

    @9 Good points Nathan. I haven’t tested this out much more yet but my workaround was to replicate exactly the same formula that’s in my computed field formula but in a formula language agent. And it all works fine now. Will look into it a bit more when I have some time.

  • http://notesin9.com David Leedy

    Kathy,
    Personally I would avoid ComputeWithForm like the plague in Notes Client Development. Oddly it seems to be a good practice for XPages stuff. go Figure.

    I would try to use LotusScript as much as possible. Rather then Input Translations and Validations I do all that in Script. Then you have the chance to reuse it in agents or libraries. And if you go far enough to create a custom class out of your business object – that’s where the gravy is for LotusScript. Now you can really reuse code where ever needed.

    Just my 2 Cents…

  • Colin Williams

    A few months back I had an agent using CWF stop working (no errors, just no longer CWF). After much head scratching I had no alternative but to get my hands dirty and write methods for all the calculations. It took me longer but was very satisfying. If you ask me, its CWF that is hack-tastic. Have fun! :)

  • http://jonvon.net John Vaughan

    Yes, workaround #2. I was trying to say #2 isn’t a workaround. It’s the way to do it.

  • http://www.lotusguru.com Kevin Pettitt

    I seem to recall Computewithform not behaving as expected many releases ago, but in the 7+ time frame I have had pretty good experience with it. With one exception.

    If your form design uses some computed for display fields, and the values of those are used in formulas for other *computed* fields, Computewithform fails. Not surprising perhaps, until you realize that it *does* work if you refresh with @Command([ToolsRefreshSelectedDocs]).

    That said, I would agree with those who advise against using this method in any kind of complex form situation. I actually am really interested in @2 Emily’s DXL approach and would love to see something like that on OpenNTF.

  • Rob McDonagh

    ComputeWithForm is the workaround, to me. Hack-tacular, too. The proper (but harder) method is what Nathan and David suggested. Like Kevin, though, I’m very interested in Emily’s technique. You might be able to do decent error handling that way, depending on the formulas in question. Hm, so far I’ve said “+1″ to 4 different comments. I clearly have nothing original to say. lol

  • http://interfacematters.com Chris Blatnick

    Another vote for calculating it yourself rather than using ComputeWithForm. That’s all…I don’t really do this stuff anymore. Emoticon

  • http://www.poweroftheschwartz.com Richard Schwartz

    Here, by the way, is a link to a case I encountered several years ago, in which an agent that did a computeWithForm caused a server crash! It caused a computed field formula to run when it didn’t actually have to, and that formula used an @dblookup option that crashed servers when it was executed in background agents.

    { Link }

  • neonsonik

    Hello. Since the upgrade on 8.5.2 I have this bug in my agents as well. I’m getting my documents with:
    Set doc=view.GetDocumentByKey(key, true)
    and next line:
    Call doc.ComputeWithForm(False, False)
    doesn’t work anymore. But suddenly, I’ve realized, that when I read smth. from doc, before ComputeWithForm call, it will help! So, next snippet works for me:

    Set doc=view.GetDocumentByKey(key, True)
    Dim bug as String
    bug=doc.UniverslaID ‘(or any item)
    Call doc.ComputeWithForm(False, False)

    Maybe this could help you as well.

  • http://www.aofoundation.org Christoph Nötzli

    I use both options (computewithform and field specific formulas) in agents to update fields in semi relational database constructs on a regular basis. I never had many problems with these constructs until 8.5.2 where the computewithform does really strange things. I tried to debug an “old” agent, because I wanted to know, where the agent failed in the new environment and detected that the computewithform statement worked fine as long as I was debugging step by step and worked no longer when I clicked on continue to run the agent as long as no error would occur (which in my case was to the end). It seems to me that this bug is unlikely to be reproducible in debugging at all, which makes it difficult to really find a solution to the problem.

    The workaround presented by neonsonik worked very well for me, but can not be a longterm solution for a bug like this. This workaround and my experience with the debugging point to the fact that the handle on the document is not correctly initiated by the agent as long as no item is queried or changed (or anything else which does something on the document) and therefore, computewithform just does nothing (which makes the agent fast but useless). Such unpredictable behavior of the newest release makes me worried about other not yet reported bugs with very difficult reproducibility.

    I hope, some lotus developer look into this as soon as possible and do their job!!

    Emoticon

  • Daniel Bjarsch

    When I have code that uses ComputeWithForm to “refresh” documents in the background, I create a new form, containing just the fields that needs to be recalculated. Then I switch form on the document, do the ComputeWithForm, and then save the document with the original form again. Usually works fine, though you end up with extra forms in your application…

  • http://null Devon Christensen

    The two useless lines of code that Richard Schwartz mentioned worked! I added those two lines right before the ComputeWithForm line and it’s fixed. Thanks, Richard!Emoticon

    Dim bug as String

    bug=doc.UniverslaID ‘(or any item)

    Call doc.ComputeWithForm(True, False)

  • http://null Stefan

    neonsonik – thanks for your post [19 { Link } ] worked like a charm.

    I had new code to computeWithForm from within a scheduled agent and it only worked when I was debugging. Needless to say I have less hair now; however, your code prevented me from pulling out all my hair Emoticon

    Thanks!

  • http://mdm-adph.blogspot.com mdmadph

    Aye, the fix in #19 is nice — any reason why this wouldn’t work to recalculate Authors fields, though? Emoticon

  • Torsten

    Thanx a lot… I faced that problem yesterday and almost got crazy, until I found this post…

    Just one more remark: Not only reading something from the doc, also writing something to it works as a workaround.

    So if you have anything like

    doc.Something = “Something”
    or
    Call doc.ReplaceItemValue( “Something” , “Something” )

    before the computewithform, then it will work as well. The only strange thing is: It seems to compute the document twice. A “Editcounter” (not a real use case, just to show the effect) would be broken: A Field with the Formula @ThisValue + 1 will always increase by 2 on each computewithform… Meanless to say, that it only increases by 1 with ToolsRefreshSelectedDocs…

    Best regards
    Torsten

  • Paul Davies

    we discovered that computewithform fails if a field with a @dblookup returns [failsilent]

    if the dblookup is rewritten back to the old fashioned method x := @dblookup(); @if(@iserror(x);”";x) then it works fine

  • Mat

    Fix List Entry:
    { Link }

    Problem Ticket:
    { Link }