• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member
  • Login

Webscraping HTML elements from Paged Websites|VBA & Macros|Excel Forum|My Online Training Hub

You are here: Home / Webscraping HTML elements from Paged Websites|VBA & Macros|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumVBA & MacrosWebscraping HTML elements from Page…
sp_PrintTopic sp_TopicIcon
Webscraping HTML elements from Paged Websites
Avatar
Kai Zen

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
December 21, 2019
sp_UserOfflineSmall Offline
1
December 21, 2019 - 8:06 am
sp_Permalink sp_Print

Hello, I am working on improving a Legislative Review project of all Legislative Bills to track the Last Action on each bill. Currently, I have VBA written to pull the Final and Signed pdf documents into a folder for an advanced search of relevant Legislative terms. What I am working on now is the last piece - to pull in the Last Action element (perhaps the title also if I get this working) on each bill so they can be tracked efficiently. I have attached a copy of what I have created to date which is the auto open General Assembly page and the code to ensure it runs once the page is loaded. The remainder of what I have is a hodge podge of what I have pulled together but I did just download the example from Web Scraping Paged Websites. I would really like to get some input into whether extracting one or two elements is possible. I cannot actually find blog out there to extract an element from paged websites. Thanks for any help from all you masterminds! Wink[Image Can Not Be Found]

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1550
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
2
December 21, 2019 - 9:21 am
sp_Permalink sp_Print

Hi Kai,

Nothing is attached 🙁

Please attach a workbook with code rather than an image of that code.  

Regards

Phil

Avatar
Kai Zen

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
December 21, 2019
sp_UserOfflineSmall Offline
3
December 21, 2019 - 9:33 am
sp_Permalink sp_Print

Oh sorry! Thank you for clarifying. I will add the file for you. I have added the image of the element needed on a blank tab in the workbook. Thanks for any feedback!

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1550
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
4
December 21, 2019 - 4:37 pm
sp_Permalink sp_Print sp_EditHistory

Hi Kai,

You can extract as many elements as you want from a web page.  The issue is how easy that is to do, and that depends on how the web page is constructed.

If the web page elements are all given an ID then it is pretty straight forward. As all ID's in a web page should be unique, knowing an ID means you know exactly how to locate the piece of information you want.

Unfortunately not all elements are given ID's (it's not a requirement when making a page) and this is the case for the web site you are looking at.  So you need to locate the info you want in another way and that can be by using the CSS class, which is how I've done it for you - see attached file.

In the image you included in your workbook, you'll see that the highlighted line has a section class="bill-last-action search-result-single-item".  This element has 2 classes bill-last-action and search-result-single-item.  I used bill-last-action to pick out the information you want using this line of code

    HTML.getElementsByClassName("bill-last-action")(ResultNum).getElementsByTagName("span")(0)

This is used in a loop where ResultNum is a loop counter going through each of the 25 results per page and the last bit .getElementsByTagName("span")(0) picks out the text in the <span> tags.

My code writes the Last Action to the Immediate Window using Debug.Print so you just need to change that code to do what you want with the text.

I've written a few posts on web scraping which might help you:

https://www.myonlinetraininghu.....g-with-vba

https://www.myonlinetraininghu.....iple-pages

https://www.myonlinetraininghu.....ling-forms

TBH I prefer using Selenium as I find it it is much easier to manipulate the web page and I'd encourage you to give that a go.  I find the native VBA syntax a bit difficult to use and not well documented 🙁

Cheers

Phil

Avatar
Kai Zen

Active Member
Members
Level 0
Forum Posts: 3
Member Since:
December 21, 2019
sp_UserOfflineSmall Offline
5
December 23, 2019 - 1:36 am
sp_Permalink sp_Print

Thank you for the great feedback. That was my determination and have downloaded Selenium but wanted to verify. I will give it a go. Thanks again!

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1550
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
6
December 23, 2019 - 12:17 pm
sp_Permalink sp_Print

No worries, good luck.

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Mynda Treacy, Syed Hussain, Lorna Henning
Guest(s) 3
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 880
Purfleet: 414
Frans Visser: 346
David_Ng: 306
lea cohen: 237
Jessica Stewart: 219
A.Maurizio: 213
Aye Mu: 201
Hans Hallebeek: 185
Newest Members:
Appiagyei Kofi Frimpong
Hilary Burchfield
Richie Wright
Adel Kock
Barbara Murray
Doug Milne
Siobhan Stringer
Rob Rooth
Tom Lewis
Jennifer Rodriguez-Avila
Forum Stats:
Groups: 3
Forums: 24
Topics: 6542
Posts: 28641

 

Member Stats:
Guest Posters: 49
Members: 32829
Moderators: 2
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel Office Scripts
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook

Sign up to our newsletter and join over 400,000
others who learn Excel and Power BI with us.

 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate
  • Sponsor Our Newsletter

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.