5 kyu

Scraping: Codewars Top 500 Users

88 of 87510XL
Description
Loading description...
Fundamentals
View
AllIssues20Questions13Suggestions2Show Resolved
  • Please sign in or sign up to leave a comment.
  • snowysantacruz Avatar

    I liked this kata. The sample tests are out of date, however. The 1-based indexing threw me for a while.

  • Mednoob Avatar

    Is this kata still completable?

    I tried doing it on JS, and I got this error:

    Unhandled rejection Error: certificate has expired
    at TLSSocket.<anonymous> (_tls_wrap.js:1105:38)
    at emitNone (events.js:106:13)
    at TLSSocket.emit (events.js:208:7)
    at TLSSocket._finishInit (_tls_wrap.js:639:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:469:38)
  • dfhwze Avatar

    Description mentions "the ith ranked User" but does not specify how to rank users. Since there is a rank inside the html, and this rank isn't used, this is a real issue.

  • dfhwze Avatar

    You don't specify how to rank users. There is honor, a rank inside the html, and the oridinal position of the results.

  • dfhwze Avatar

    You don't specify the type of "position". It could be interpreted as array, not as object, as intended.

  • dfhwze Avatar

    How are these tests even made? I pass random tests all of a sudden, without passing sample tests.

  • dfhwze Avatar

    Users can have the same rank:

    vintrom #209 Valefar #209

    How do we determine rank ??????

  • dfhwze Avatar
     
    Leaderboard#position should contain 500 'User' objects.
Leaderboard#position[i] should return the ith ranked User(1 based index).
    • So what should be on position[0] ????
    • error on 500th user: Cannot read property 'name' of undefined
  • Little Turtle Avatar

    This kata in JavaScript can't be done using Cheerio only. I've done it with Puppeteer but, unfortunately, I wasn't allowed to import Puppeteer here in order to submit my solution.

  • HerrWert Avatar

    This comment has been hidden.

  • VladLozynskyi Avatar

    I got following error in this kata: /usr/local/lib/ruby/2.5.0/net/protocol.rb:44:in connect_nonblock': SSL_connect returned=1 errno=0 state=error: certificate verify failed (unspecified certificate verification error) (OpenSSL::SSL::SSLError)

  • Tsubasanut Avatar

    This comment has been hidden.

  • AntonSI Avatar

    This kata was extremely efficient and useful! Thanks!!

  • CodyJamesBrooks Avatar

    Excellent Kata.

  • user9644768 Avatar

    Markdown formatting is broken.

  • ashishra0 Avatar

    Stuck with that 1 based index for Ruby man. Need help on that :(

  • iwtga Avatar

    Great Kata @10XL! A Kata that covers various parts of the language, modules and also data structures XD

  • wdlord Avatar

    I don't understand, I can only pass all the cases on the attempt if I return a leaderboard with size 501 but I get an out of bounds error with size 500. How do I return a list of size 500 with 500 objects but the indexes have to start at 1?

    Also, Im getting this error:

    The code that caused this warning is on line 52 of the file main.py. To get rid of this warning, change code that looks like this:

    BeautifulSoup(YOUR_MARKUP})

    to this:

    BeautifulSoup(YOUR_MARKUP, "html.parser")

    markup_type=markup_type))

    my code looks like this: soup = BeautifulSoup(page.content, "html.parser") why is that wrong?

    Thanks

  • RevSLightfoot Avatar

    In case somebody reads the comments before trying this Kata - the test files reference out-dated stats and thus make this impossible to pass!!!

  • cutlerheman Avatar

    I did this Kata using only built-in python modules. When attempting the Kata, I ran into very strange encoding issues. On some attempts, everything would match up fine and others, I would end up with values like "\xe4\xb8\xad\xe5\x9b\xbd". I ended up hardcoding a few of the non standard characters which allowed the Kata to complete successfully after a few random attempts. I would be interested to know if anyone knows why the random encoding/decoding occurs?

  • user8436785 Avatar
  • v-h-z Avatar

    test is running green but attempt gives me this : main.rb:65:in process_data_sol': undefined method map' for nil:NilClass (NoMethodError) from main.rb:74:in build_leaderboard' from main.rb:81:in

    ' What am I doing wrong ? :( Thanks !

  • spodek Avatar

    obsolete. ranking has changed. Tests failed :(

  • evgeniy_pp Avatar

    This comment has been hidden.

  • FArekkusu Avatar

    If you're asking us to scrape 500 users from the leaderboard, then you should test all of them, not 150 (in the best case scenario).

  • FArekkusu Avatar

    The leaderboard is a list. It makes sense to use a list. Now, why on earth are you forbidding its use by forcing 1-indexing and asking for the answer length to be 500? I realized that I could have used a dict instead after seeing other people's solution, but that's overkill in this case.

  • FArekkusu Avatar

    Sample tests don't correspond to the actual state of the leaderboard.

  • evgeniy_pp Avatar

    This comment has been hidden.

  • jtokib Avatar

    This comment has been hidden.

  • vsakun Avatar

    in Fixed Tests, if there is no clan, you must output 'None', and in Randomized Tests, if there is no clan, instead of 'None' show

    None should equal ''

    or am I misunderstanding something?

  • pinkston4 Avatar

    Python:

    Indexing on 1...Give me something to google please!? I am starting to feel like the code challange is less about the scraping (which is simple and takes like 5 min) and more about that. Or am I way over thinking this?

  • VCrawford952 Avatar

    My solution (in jupyter notebooks in my local environment, used lxml. But when I try in the CodeWards environment i get an import error

    import lxml

    ImportError: No module named 'lxml'

    Any ideas for how I get round this?

  • user5495976 Avatar

    Well, and unlike most of the comments here, I worked on this kata to practice OOP as I just learned about it 3 days ago!😁

  • qq1326678957 Avatar

    I submitted my code time and time again.10 minutes later，I finish it.so you need to wait.

  • Anatolych97 Avatar

    Good kata. It was interesting, but test cases is very uncomfortable. In the first time, i turned off them that to see my console.log()

  • sam43125 Avatar

    This comment has been hidden.

  • 0xjgv Avatar

    The leaderboard has changed!

  • ticktricktrack Avatar

    The forced in hash access for position[n] is really annoying, why not make it a method position(n). I basically had to add wrap Array in a class that sutracts 1 from the index. And why test against the live site, a frozen copy of the leaderboard would work just as well.

  • DocMors Avatar

    Very funny! I run the script from my box and all is working as expected then I try to run it through the website and it fails. After 10 minute troublshooting the issue it turns out that the codewars LDB page just now decided to give up the ghost!

  • davedema Avatar

    This comment has been hidden.

  • kclaw99 Avatar

    Bravo!!!! Well done, this was a great kata. I learned a lot, had never done any web scrapping, so bit of a right of passage. If you are thinking of whether to do this Kata, I could not recommend it more. Also looking at the Kata test cases after solving it is well worth your time, some great code.

  • suic Avatar

    Thanks, this was fun! Regards, suic

  • bestwebua Avatar

    @10XL, awesome kata! Big thanks for mentoring and test cases fixes. Cheers!)

  • Blind4Basics Avatar

    This comment has been hidden.

  • bestwebua Avatar

    Is this right way for Ruby?

    class User
  #some code
end

class Leaderboard
  #some code
end

def solution
  Leaderboard.position
end

    I have got a lot of errors like below, however locally everything works okay:

    #>
main.rb:39:in `block (2 levels) in 
'
/runner/frameworks/ruby/cw-2.rb:180:in `wrap_error'
/runner/frameworks/ruby/cw-2.rb:72:in `it'
/runner/frameworks/ruby/cw-2.rb:206:in `it'
main.rb:38:in `block in 
'
/runner/frameworks/ruby/cw-2.rb:55:in `block in describe'
/runner/frameworks/ruby/cw-2.rb:46:in `measure'
/runner/frameworks/ruby/cw-2.rb:51:in `describe'
/runner/frameworks/ruby/cw-2.rb:202:in `describe'
main.rb:37:in `
'

    Thanks for the help!

  • 10XL Avatar

    JS translation is broken/unsupported.

  • Firefly2002 Avatar

    This comment has been hidden.

  • krelly Avatar

    Sample tests for ruby is invalid. Test are failing because open(url) performs the actual network request, getting the most recent data and tests is based on old data. Also if you try to submit solution (by pressing attempt button) even if sample test failing, it will be accepted (if it working properly), this is very confusing.

    I think the solution should be to stub network requests or use hidden tests (i'm not sure however is it possible to hide the body of process_data_sol and build_leaderboard from user until he solves this kata).

  • HerrWert Avatar

    Can someone point me to a good online resource for using jsdom as required in this kata?

  • apostonaut Avatar

    How can I handle the 1-based indexing? The second element of the position list has to be the first on the leaderboard, and the 500th element must be the 501st on the list but there can only be 500 in the list? I don't see any way around this

  • mmalkavian Avatar

    Python:

    How can I deal with characters in clans like

    (╯°□°）╯︵ ┻━┻

    except doing like the emoji suggests? :D

  • kurt343 Avatar

    Hi 10XL,

    I think there may be another issue with your tests:

    Expected: 'Founders & Coders', instead got: 'Founders & Coders'

    "Founders & Coders" is how this particular clan name displays on the leaderboard page. This is preventing me from completing the kata.

    Or is this intentional?

  • buttonupbub Avatar

    Thanks for this novel kata. For JavaScript, cheerio would be nice to have as an alternative to jsdom.

  • croceflamenco Avatar

    I really enjoyed this kata. Thank you =]

  • zruF Avatar

    Wow, great Kata! I've learned a lot! Thanks :)

  • daddepledge Avatar

    What data structure should the python code return?

  • adamdharrington Avatar

    Really cool stuff, I hadn't looked at Nokogiri's API properly before so I'm really glad you made this!

  • camerican Avatar

    Hi 10XL,

    Any idea about the following error within #build_leaderboard? It appears to be on your end within the test suite.

    `initialize': wrong number of arguments (given 3, expected 1) (ArgumentError)
    from `new'
    from `block in build_leaderboard'
    from  `map'
    from  `build_leaderboard'
    from  `<main>'
  • smile67 Avatar

    There are other cases too which don't work. My list contains exaxtly 500 entries (as long as the leaderboard), but some are not at same place compared with your list (it's difficult to recognize the exact places, because your object misses position entries visible for tests). One actual example is:

    ✘ Expected: 1352, instead got: 1353
✘ Expected: 2902, instead got: 2905
✔ Test Passed: Value == 1396
✔ Test Passed: Value == 1264

    If i open the leaderboard i only see an entry of 1353, but no entry contains honor points of 1352 (what you expect). So it's not possible to pass the tests. By the way codewars sometimes shows older boards, some minutes later refreshed boards - it changes sometimes (hope you know what i mean)...

  • smile67 Avatar

    This comment has been hidden.