New HIT Monitor Userscript and AHK Script

Discussion in 'mTurk Scripts & Extensions' started by DCI, Jan 29, 2014.

  1. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Update 11/7/14 - New Release

    If you have any questions about either of the scripts posted in this thread, feel free to leave them here. I'm subscribed to the thread and I'll get an email notification if you post here. I don't mind answering questions in the daily HIT thread, but I might never notice your question there. Also if it's here it might help other people with the same question in the future.

    New HIT Monitor userscript: http://userscripts-mirror.org/scripts/show/399555

    This is a script that I made that continuously monitors mturk for new HITs and alerts you if anything on your list is found. It works simply by reloading the search page and checking for any text that you give it. This has the advantage of allowing you to search for as many requesters or HITs as you want without increasing the amount of page requests sent and causing you to go over mturk's limit. There are detailed instructions on how to use the script on the description page.


    FAQ



    Why should I use this?


    This script is strong where Page Monitor is weak. You can watch for hundreds of different requesters and HITs and you never have to purge anything from your list to make room for new things. It also has the benefit of alerting you within a few seconds when a new HIT goes up. Sometimes there's a big difference between 5 seconds and 30+ seconds.

    Should I replace Page Monitor with this?

    I recommend keeping PM and having a small list of your most important things on it in addition to New HIT Monitor. Once in a while you might miss something due to multiple HITs going up one right after another, and also once in a while some HITs will not show up as new when they go up. By having both you can compensate for the weaknesses of each of them.

    How do I edit a script? I've never done this before.

    In Chrome:
    Left click the tampermonkey logo on your extension bar. If you don't see it, click the arrow by your settings button in the upper right of your browser. If you still don't see it, go into chrome://extensions/ and click "Show Button" for tampermonkey.

    Now click on Dashboard.

    [​IMG]

    Then click on the New HIT Monitor Script. Click the text itself, not the button on the left. That button is a toggle switch that turns the script on and off. It's green when things are on and red when they are turned off.

    [​IMG]

    In Firefox:
    Click on the arrow on the right side of the greasemonkey icon in your extension bar in the upper right hand corner of your browser. Then click on "Manage User Scripts."

    [​IMG]

    Find the New HIT Monitor script on the list, which will look like this:

    [​IMG]

    Click the options button, and then click the "Edit this User Script" button at the bottom of the box that pops up.

    Once you've opened the script, you'll see a list in there that looks something like this:

    Code:
    "ACME Data Collection",
    "agent agent",
    "AJ Ghergich",
    "Andy K",
    "BICC",
    "Bluejay Labs",
    "carnegie mellon social computing group",
    "Christos Koritos",    
    "Dan Shaffer",
    "David Mease",
    "Funicular Heavy Industries",
    "Gaddy",    
    "Heather Walters",
    "JASON W GULLIFER",
    "Jeff Foster",
    "jesse egbert",
    "Jonathan Frates",
    "Leonid",
    "nabirds",
    "nlp",    
    "OCMP",
    "Parisa",
    "Personagraph",
    "pickfu",
    "Procore", 
    "Project Endor",
    "Project Gandolph",
    "Sergey Schmidt",
    "SIRIUSProject",
    "Smartsheet",
    "Spreecast",
    "Stiglitz",
    "Tag Requester",
    "Two Lakes",
    "User Manual",
    "vaplab",    
    "Vesterman",
    "UW Social", 
    "Wharton",    
    "World Vision",
    "x8 data"
    Now just add in whatever requesters or HITs that you like in the same format and remove anything that you don't want.

    Your search terms do not have to be in any particular order. I just like to alphabetize mine because I think it makes it easier to keep track of what I have. They also do not need to be spaced in any particular way. Your list can be like this:

    Code:
    "ACME Data Collection","agent agent","AJ Ghergich","Andy K","BICC","Bluejay Labs","carnegie mellon social computing group","Christos Koritos","Dan Shaffer","David Mease","Funicular Heavy Industries","Gaddy",   
    "Heather Walters","JASON W GULLIFER","Jeff Foster","jesse egbert","Jonathan Frates","Leonid","nabirds","nlp","OCMP","Parisa","Personagraph","pickfu","Procore","Project Endor","Project Gandolph","Sergey Schmidt","SIRIUSProject","Smartsheet","Spreecast","Stiglitz","Tag Requester","Two Lakes","User Manual","vaplab","Vesterman","UW Social","Wharton","World Vision","x8 data"
    or like this:

    Code:
    "ACME Data Collection",
                  "agent agent",
    "AJ Ghergich",
              "Andy K",
    "BICC",
    "Bluejay Labs",
                "carnegie mellon social computing group",
    "Christos Koritos",    
    
    "x8 data"
    It doesn't make any difference as long as each term is in quotes and followed by a comma. The last term on the list is the only one that shouldn't be followed by a comma.

    Do not add special characters into your list like / ( ]. These can break your script.

    You can use terms other than just the requester name or the name of a HIT if you like. This can be useful for requesters with very short names like TR. The script searches the entire page for text, including all the things in a HIT's description.

    This script will also search the text inside of the hrefs in links, so you can use requester IDs or HIT IDs as search terms.

    You can add notes to your list to help you remember information about requesters or HITs. To do this, just start a line with // Anything on that line will just be a note and disregarded by the browser when it runs the script.

    Example:

    Code:
    "AJ Ghergich",
    // 9 cent find email HITs. Not bad, but be careful of rejects.
    "quality shopping",
    // OCMP HIT where you rate page of search results on Walmart's site
    My script isn't doing anything. What do I do?

    When your script isn't working, you'll know it because the page won't be reloading every 5 seconds like it's supposed to. It will just be sitting there doing nothing. There are 3 main reasons for this happening.

    1) You were recently alerted and have not restarted the script - There is a 5 minute delay in reloading after each alert that you get. This is to give you time to get information or follow links on the page as well as keeping the script from driving you insane by alerting you over and over when a batch is going up. You can simply reload the page if you'd like it to restart it yourself sooner than that.

    2) You broke the script - It's easy to make typos when editing your list, and that can make your script stop working. This happens a lot when people forget to add a comma after a search term or leave a quote open. Sometimes you can save yourself the time of looking through your entire list by opening the script in the tampermonkey editor and clicking the syntax check button. This will usually highlight a spot on your list right after where you made a typo. Also, you may have broken your script by having a search term with special characters like ( / ]. Don't put those in your search terms.

    3) You're loading the wrong link - This script is restricted to running at one specific url. If you're not at that url, it won't run. To check and see if your script is running, load your New HIT Monitor bookmark and then left click the tampermonkey or greasemonkey icon in your extensions bar. It will list the scripts that are running on the current page. If you don't see your script, then the url you're at doesn't match the url in the @include line at the top of the script. You can always go back to the description page of the userscript to get the link again if you need it. Also note that tampermonkey and greasemonkey both have script toggles. They will still show what scripts run on a page even if they are turned off though. In tampermonkey there are red and green buttons to display activation, and in greasemonkey checkmarks signify activated scripts.

    Okay, I did everything just like you said, but I'm confused. It seems to be working, but what does this thing do again? I see HITs, but these aren't on my list. I don't hear any alerts. I just don't get it.

    All this script does is loads a page, looks for words, and then let's you know when it finds those words. The page it looks at is the mturk search page, sorted by newest HITs. All it's doing is watching the new HITs that get posted and watching for your words. Most of the time they won't be there and it will just be a page full of Oscar, Brelig, Crowdsource, etc. Just leave that tab running and it will let you know when the things on your list pop up. I recommend pinning the tab so that it takes up less space and doesn't get closed accidentally. Also, if you want to see what the alert looks like when something does pop up, just add something to your list that will always be on the page, like your name or the word "results." Then you can see for yourself what will happen when one of your HITs pops up.

    AHK version of New HIT Monitor: https://dl.dropboxusercontent.com/u/353548/HITmonitor.zip

    This is an AHK version of New HIT Monitor that works in the same way that the userscript does. The main difference between the two is that the AHK version runs in the background while the userscript version runs in a tab. Most people prefer the userscript version, and I do too. The AHK version is a little bit more user friendly when it comes to editing your requester list, but it can also be a little bit buggy at times and produce error messages.

    To use the script, first you must have AHK installed. Download and extract the zip file linked above. Then just double click the New HIT Monitor script to run. To edit your list, right click the script and Edit. The requester list is right at the top and you simply type in whatever you'd like to search for. Then save and reload the script.

    If the fisrt time running this you get "Error 0x800704a6", it will sometimes be fixed by rebooting your machine.

    If this continues to happen, you need make sure that Windows is up to date, particularly IE and net framework. You should also make sure that you have the most recent version of AHK. This script won't work on some older versions.

    If you're still getting errors, and this is common in Windows XP particularly, right click the script and compile it into an exe. Then make sure that you run the exe as administrator. This gets a little annoying if you edit your list a lot, but it works pretty well.

    Similar to the userscript, this has a rest function so that it doesn't drive you crazy when a batch on your list is going up for a long time. When you click the OK button on an alert, the script will restart immediately. If you don't want that, just click your browser instead and leave it in the background. It will restart itself after a couple of minutes in case you forget.
     
    • Like Like x 41
  2. Colie Annie New Member Member

    Joined:
    Nov 26, 2013
    Messages:
    4
    Likes Received:
    12
    I'm not sure if I've missed this but is there a way to disable the pop up and rely solely on the noise to alert for changes? If I'm working in a quick moving batch the pop up maybe a hinderance if I'm very short on time. Thank you btw this is a very useful script.
     
    • Like Like x 1
  3. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Yeah, it's pretty easy to remove the popup. Just look for the line right under where it says mcoinsound.

    Code:
    setTimeout(function(){alert("Alert" + found)}, 2000);
    That's the line that makes the popup. You can remove that or temporarily make it a note and it'll still make the sound when something comes up.

    Edit: I should probably also add that the alert behavior is a little different in Firefox than it is in Chrome. In Chrome you're forced to OK the dialogue to unlock any browser functions, but in FF you can still switch to other tabs and do anything else while an alert is up.
     
    • Like Like x 5
  4. Colie Annie New Member Member

    Joined:
    Nov 26, 2013
    Messages:
    4
    Likes Received:
    12
    I can see that alert being easier to work around in Firefox in that case then. In chrome I was running on very short timers that left me 30 seconds to do each task. I have a couple requestors like that so cutting it back to just an audible alert saves precious seconds. Removing the line you told me to worked perfectly. Thank you :)
     
    • Like Like x 1
  5. clickhappier ★★Ⰼ₳ՖŦξᚱ⌚ Contributor

    Joined:
    Jul 1, 2014
    Messages:
    5,600
    Likes Received:
    9,527
    After discussion with DCI, I've posted his New HIT Monitor userscript, and a modified version of it by inactive turker 'mralaska', in my Greasyfork archive for easier availability. :)

    DCI New HIT Monitor for mturk (classic version)
    DCI New HIT Monitor for mturk (with links) (mralaska version)

    Mralaska also made a variation of it for monitoring the CrowdSource WorkStation site:
    Crowdsource Workstation Monitor
     
    • Like Like x 2
  6. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    I made a new and improved version of this today that I think is pretty nifty. This one cycles through the first 3 pages of new HITs so that you don't have any issues with missing things due to a log jam of new posts. I also integrated the Mralaska linkmode mod into this version so that instead of using a pop up alert, it grabs the links from the HITs that it finds and writes them into a new tab/popup. This version works really well logged off or in private browsing window. It doesn't work in Chrome at the moment.

    https://greasyfork.org/en/scripts/6316-dci-new-hit-monitor-party-edition-firefox
     
    • Like Like x 6
  7. zingy RebΦrn Contributor

    Joined:
    Oct 18, 2013
    Messages:
    38,201
    Likes Received:
    101,298
    Good stuff DCI. I always a fan of the old you posted last year. This one should be great.
     
    • Like Like x 3
  8. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    I never really looked through that Mralaska mod script before today, but the linkmode that I grabbed from it is pretty awesome. He made it so that it also grabs the time and date that searches are found and also writes that to the alert page. You could use that to keep a complete log of anything and everything that posts if you wanted to.

    Also, I didn't realize back when I made the original that it will find search terms in the hrefs of links, so you can use requester IDs or HIT IDs as search terms.
     
    • Like Like x 3
  9. clickhappier ★★Ⰼ₳ՖŦξᚱ⌚ Contributor

    Joined:
    Jul 1, 2014
    Messages:
    5,600
    Likes Received:
    9,527
    Working well in Firefox (Pale Moon). :thumb: One point for improvement: I think the search results tab could scroll to the bottom of itself automatically when new stuff gets added, I'm having to do that manually currently.
     
    • Like Like x 2
  10. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Yeah, I agree. I'm not exactly sure how to do it though. It might be easier to make it write to the top, although I'm not exactly sure how to do that either xD I'll try to get it figured out though.
     
    • Like Like x 1
  11. clickhappier ★★Ⰼ₳ՖŦξᚱ⌚ Contributor

    Joined:
    Jul 1, 2014
    Messages:
    5,600
    Likes Received:
    9,527
    This should help. :)
     
    • Like Like x 2
  12. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Thanks. I've used scroll before, but never on a popup before and also wasn't sure it would work if the window was already open. That was a lot easier than I thought it would be xD

    Updated and should always scroll to the bottom on new alerts now.

    Edit: oops, nm. that did work, but also broke the script x_x. take 2
     
    • Like Like x 2
  13. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Ok, fixed for real this time and updated.
     
    • Like Like x 2
  14. clickhappier ★★Ⰼ₳ՖŦξᚱ⌚ Contributor

    Joined:
    Jul 1, 2014
    Messages:
    5,600
    Likes Received:
    9,527
    Yep, that did it. :nod:
     
    • Like Like x 2
  15. Red New Member Member

    Joined:
    Nov 8, 2014
    Messages:
    9
    Likes Received:
    12
    Hello DCI. Thanks for the script. Nice improvement.

    I have a question.

    Is it possible to manipulate the search parameter URL to query only those HITs we are qualified for, and reflect a set amount? In other words if I want to search HITs I am qualified for that pay .03 minimum can I manipulate your URL in the script in order to do this? The current URL shows all HITs, even those we are not not qualified for, and for any $ amount. For example, I do not do penny HITs, so I'd like to eliminate those results. And, there are many HITs I cannot do appearing as well. I tried a couple things unsuccessfully before asking you, but they did not work.

    Thank you for any help.
     
    • Like Like x 1
  16. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Yes, it's possible, but not in a very user friendly way. To do it you need to add the 3 links that you want in the meta data at the top of the script as well as in the firstpage, secondpage, thirdpage functions in the script.

    I think that if you want to search for HITs you're qualified for over 3c, it's probably a lot more practical to just use the original version though. (To do that you only need to add the url you want in the meta data and launch from that url.) That search will filter out a lot of stuff, and so it will be very rare for it to miss something before it falls off the first page on it's default 5 second delay.

    This latest version is better designed for offline monitoring since it cycles though 3 pages. The default speed is much faster, but since it checks 3 pages it can run offline and monitor all HITs without ever missing anything. It uses more page requests, but you get 2 different page request pools. There is one for page requests from your account and one for page requests from your IP. I just run the recent version on a laptop on the side, logged out. That way I have all of my account page requests and no extra load on my main computer.

    Edit: If you want the original with the linkmode, you can use the Mralaska version that clickhappier linked.
     
    • Like Like x 1
  17. Red New Member Member

    Joined:
    Nov 8, 2014
    Messages:
    9
    Likes Received:
    12

    Ok, gotcha. It does sound more user friendly to stick with the Mralaska version, which is what I run now. I get what you said.

    Thanks. Have a good one.
     
    • Like Like x 2
  18. clickhappier ★★Ⰼ₳ՖŦξᚱ⌚ Contributor

    Joined:
    Jul 1, 2014
    Messages:
    5,600
    Likes Received:
    9,527
    Edit: Never mind, DCI's right, the search parameters look easy to add to the URLs at first, but the ugly version of the URLs that appear for subsequent pages, and that he has HIT Monitor set up to run on, don't cooperate with the pretty-looking easy search parameters.
     
    • Like Like x 2
  19. DCI $_$ Member

    Joined:
    Oct 18, 2013
    Messages:
    3,864
    Likes Received:
    13,319
    Yeah, I'd like to clean those urls up to make it easier to customize. The reason I used the next/first links is because when I just used the page number it would show results as 1-10 on any of the 3 pages, which is what I used to distinguish between them.
     
    • Like Like x 1
  20. Nickhatesyou asshat Member

    Joined:
    Mar 11, 2014
    Messages:
    5,809
    Likes Received:
    15,469
    nevermind figured it out
     
    • Like Like x 1

Share This Page