• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Is a cookie pop-up the problem?|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Is a cookie pop-up the problem?|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryIs a cookie pop-up the problem?
sp_PrintTopic sp_TopicIcon
Is a cookie pop-up the problem?
Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
1
November 9, 2022 - 9:30 am
sp_Permalink sp_Print

Hi folks, I've done the PQ course and have a few more undergoing now, and I love them!

I have a small query I wrote for my son-in-law. Worked fine for 2-years. Now it fails with the message: can't find column 'lifetime record', which is daft, because I can see it. But I think the problem may be the 'new' cookie pop-up that blocks advancing until it is answered, and I don't know how to answer from a PQ.

Any ideas would help a lot. It's only a hobby app - but this is really enjoyable to me, so thanks for keeping me from pruning the roses.

 

Here's a 1-horse version of the query:

//(startpage as text)=>
let
//Source = Web.Page(Web.Contents("https://www.racingpost.com/profile/horse/"&startpage&"/form")),
Source = Web.Page(Web.Contents("https://www.racingpost.com/profile/horse/1888963/shishkin/form")),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"lifetime record", type text}, {"runs", Int64.Type}, {"wins runs", type date}, {"2nds", Int64.Type}, {"3rds", Int64.Type}, {"winnings", type text}, {"prize", type text}, {"earnings", type text}, {"or", type text}, {"best ts", type text}, {"best rpr", type text}, {"best mr", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([lifetime record] <> "PTP" and [lifetime record] <> "Rules")),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"lifetime record", "or", "best ts", "best rpr"}),
#"Replaced Value" = Table.ReplaceValue(#"Removed Other Columns","—","0",Replacer.ReplaceText,{"or", "best ts", "best rpr"}),
#"Added Conditional Column" = Table.AddColumn(#"Replaced Value", "Chase OR", each if [lifetime record] = "Chase" then [or] else null),
#"Added Conditional Column1" = Table.AddColumn(#"Added Conditional Column", "Chase Best TS", each if [lifetime record] = "Chase" then [best ts] else null),
#"Added Conditional Column2" = Table.AddColumn(#"Added Conditional Column1", "Chase Best RPR", each if [lifetime record] = "Chase" then [best rpr] else null),
#"Added Conditional Column3" = Table.AddColumn(#"Added Conditional Column2", "Hurdle OR", each if [lifetime record] = "Hurdle" then [or] else null),
#"Added Conditional Column4" = Table.AddColumn(#"Added Conditional Column3", "Hurdle Best TS", each if [lifetime record] = "Hurdle" then [best ts] else null),
#"Added Conditional Column5" = Table.AddColumn(#"Added Conditional Column4", "Hurdle Best RPR", each if [lifetime record] = "Hurdle" then [best rpr] else null),
#"Added Conditional Column6" = Table.AddColumn(#"Added Conditional Column5", "NHF Best TS", each if [lifetime record] = "NHF" then [best ts] else null),
#"Added Conditional Column7" = Table.AddColumn(#"Added Conditional Column6", "NHF Best RPR", each if [lifetime record] = "NHF" then [best rpr] else null),
#"Removed Other Columns1" = Table.SelectColumns(#"Added Conditional Column7",{"Chase OR", "Chase Best TS", "Chase Best RPR", "Hurdle OR", "Hurdle Best TS", "Hurdle Best RPR", "NHF Best TS", "NHF Best RPR"}),
#"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Removed Other Columns1", {}, "Attribute", "Value"),
#"Changed Type1" = Table.TransformColumnTypes(#"Unpivoted Columns",{{"Value", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[Attribute]), "Attribute", "Value", List.Sum)
in
#"Pivoted Column"

And if you have time to help, thanks so much, it's really appreciated.

All best wishes,

Simon

sp_AnswersTopicSeeAnswer See Answer
Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1518
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
2
November 9, 2022 - 12:07 pm
sp_Permalink sp_Print

Hi Simon,

The webpage appears to have changed since you wrote that query.  Some column names have changed e.g. Wins instead of Win Runs, and some columns have been removed e.g. Prize.

I'd say the site has been redesigned and is now rendering tables using JavaScript, rather than static HTML.  That's the reason you can't see the table any more with this query.  PQ is unable to interact with or see tables created with JavaScript.

You can try using Power Automate to get the table, or VBA.

Regards

Phil

sp_AnswersTopicAnswer
Answers Post
Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
3
November 9, 2022 - 6:00 pm
sp_Permalink sp_Print

Phil & Mynda,

You are amazing! Thanks so much for your answer to my question, that saves me from beating my head against a brick wall.

Are you thinking of doing a Power Automate course?

Kind regards & all best wishes,

Simon

Avatar
Riny van Eekelen
Örnsköldsvik, Sweden
Moderator
Members


Trusted Members

Moderators

Power BI
Level 0
Forum Posts: 441
Member Since:
January 31, 2022
sp_UserOfflineSmall Offline
4
November 9, 2022 - 7:25 pm
sp_Permalink sp_Print

You might want to try PQ that's part of PowerBI. It has a more advanced way to connect to web-sites. I took your link and could extract a table that has the headers you have in your query. The table is very small though, and I'm not even sure it's the correct one.

The picture below shows the data that can be produced based on your query (though I did remove the last pivot step).

Screenshot-2022-11-09-101708.pngImage Enlarger

I've also attached the PowerBI file (.pbix)

sp_PlupAttachments Attachments
  • sp_PlupImage Screenshot-2022-11-09-101708.png (41 KB)
Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
5
November 10, 2022 - 9:54 pm
sp_Permalink sp_Print

Thanks Riny, that's really helpful. I shall give that a try and report back if I make progress. 

It won't be for a couple of days, there has been some PQ work dropped in my in-box!

 

All best wishes and many thanks,

Simon

Avatar
Simon Smith
Member
Members
Level 0
Forum Posts: 26
Member Since:
May 19, 2021
sp_UserOfflineSmall Offline
6
November 24, 2022 - 12:02 am
sp_Permalink sp_Print

Riny, Phil,

Riny, that was a brilliant suggestion! I started a query in PowerBI to get it to find the source and then copy/paste the rest of my scripts from Power Query into PBI. It worked! I tried it the other way round but the Html.Table() function didn't seem to work in PQ. I have no idea how anyone could possibly understand the automatically created function, which I attach for amusement. It is the ugliest piece of code I have ever seen and I first started coding on Algol60 in the early 70s!

But it works, so I'm as happy as Larry - many thanks.

Simon

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Velouria, Nada Perovic
Guest(s) 10
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 871
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
Raj Mattoo
Mark Luke
terimeri dooriyan
Jack Aston
AndyC
Denise Lloyd
michael serna
mashal sana
Tiffany Kang
Leah Gillmore
Forum Stats:
Groups: 3
Forums: 24
Topics: 6219
Posts: 27279

 

Member Stats:
Guest Posters: 49
Members: 31903
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.