Hi folks, I've done the PQ course and have a few more undergoing now, and I love them!
I have a small query I wrote for my son-in-law. Worked fine for 2-years. Now it fails with the message: can't find column 'lifetime record', which is daft, because I can see it. But I think the problem may be the 'new' cookie pop-up that blocks advancing until it is answered, and I don't know how to answer from a PQ.
Any ideas would help a lot. It's only a hobby app - but this is really enjoyable to me, so thanks for keeping me from pruning the roses.
Here's a 1-horse version of the query:
//(startpage as text)=>
let
//Source = Web.Page(Web.Contents("https://www.racingpost.com/profile/horse/"&startpage&"/form")),
Source = Web.Page(Web.Contents("https://www.racingpost.com/profile/horse/1888963/shishkin/form")),
Data0 = Source{0}[Data],
#"Changed Type" = Table.TransformColumnTypes(Data0,{{"lifetime record", type text}, {"runs", Int64.Type}, {"wins runs", type date}, {"2nds", Int64.Type}, {"3rds", Int64.Type}, {"winnings", type text}, {"prize", type text}, {"earnings", type text}, {"or", type text}, {"best ts", type text}, {"best rpr", type text}, {"best mr", type text}}),
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each ([lifetime record] <> "PTP" and [lifetime record] <> "Rules")),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"lifetime record", "or", "best ts", "best rpr"}),
#"Replaced Value" = Table.ReplaceValue(#"Removed Other Columns","—","0",Replacer.ReplaceText,{"or", "best ts", "best rpr"}),
#"Added Conditional Column" = Table.AddColumn(#"Replaced Value", "Chase OR", each if [lifetime record] = "Chase" then [or] else null),
#"Added Conditional Column1" = Table.AddColumn(#"Added Conditional Column", "Chase Best TS", each if [lifetime record] = "Chase" then [best ts] else null),
#"Added Conditional Column2" = Table.AddColumn(#"Added Conditional Column1", "Chase Best RPR", each if [lifetime record] = "Chase" then [best rpr] else null),
#"Added Conditional Column3" = Table.AddColumn(#"Added Conditional Column2", "Hurdle OR", each if [lifetime record] = "Hurdle" then [or] else null),
#"Added Conditional Column4" = Table.AddColumn(#"Added Conditional Column3", "Hurdle Best TS", each if [lifetime record] = "Hurdle" then [best ts] else null),
#"Added Conditional Column5" = Table.AddColumn(#"Added Conditional Column4", "Hurdle Best RPR", each if [lifetime record] = "Hurdle" then [best rpr] else null),
#"Added Conditional Column6" = Table.AddColumn(#"Added Conditional Column5", "NHF Best TS", each if [lifetime record] = "NHF" then [best ts] else null),
#"Added Conditional Column7" = Table.AddColumn(#"Added Conditional Column6", "NHF Best RPR", each if [lifetime record] = "NHF" then [best rpr] else null),
#"Removed Other Columns1" = Table.SelectColumns(#"Added Conditional Column7",{"Chase OR", "Chase Best TS", "Chase Best RPR", "Hurdle OR", "Hurdle Best TS", "Hurdle Best RPR", "NHF Best TS", "NHF Best RPR"}),
#"Unpivoted Columns" = Table.UnpivotOtherColumns(#"Removed Other Columns1", {}, "Attribute", "Value"),
#"Changed Type1" = Table.TransformColumnTypes(#"Unpivoted Columns",{{"Value", Int64.Type}}),
#"Pivoted Column" = Table.Pivot(#"Changed Type1", List.Distinct(#"Changed Type1"[Attribute]), "Attribute", "Value", List.Sum)
in
#"Pivoted Column"
And if you have time to help, thanks so much, it's really appreciated.
All best wishes,
Simon
Hi Simon,
The webpage appears to have changed since you wrote that query. Some column names have changed e.g. Wins instead of Win Runs, and some columns have been removed e.g. Prize.
I'd say the site has been redesigned and is now rendering tables using JavaScript, rather than static HTML. That's the reason you can't see the table any more with this query. PQ is unable to interact with or see tables created with JavaScript.
You can try using Power Automate to get the table, or VBA.
Regards
Phil
Phil & Mynda,
You are amazing! Thanks so much for your answer to my question, that saves me from beating my head against a brick wall.
Are you thinking of doing a Power Automate course?
Kind regards & all best wishes,
Simon
You might want to try PQ that's part of PowerBI. It has a more advanced way to connect to web-sites. I took your link and could extract a table that has the headers you have in your query. The table is very small though, and I'm not even sure it's the correct one.
The picture below shows the data that can be produced based on your query (though I did remove the last pivot step).
I've also attached the PowerBI file (.pbix)
Thanks Riny, that's really helpful. I shall give that a try and report back if I make progress.
It won't be for a couple of days, there has been some PQ work dropped in my in-box!
All best wishes and many thanks,
Simon
Riny, Phil,
Riny, that was a brilliant suggestion! I started a query in PowerBI to get it to find the source and then copy/paste the rest of my scripts from Power Query into PBI. It worked! I tried it the other way round but the Html.Table() function didn't seem to work in PQ. I have no idea how anyone could possibly understand the automatically created function, which I attach for amusement. It is the ugliest piece of code I have ever seen and I first started coding on Algol60 in the early 70s!
But it works, so I'm as happy as Larry - many thanks.
Simon
