Log In
Register
Home
April 2, 2021
Offline
Thanks for the great post.
I am extracting data (through power query) from multiple .csv files. These individual .csv files (262 of them) are dynamically updated from one of the trading platform.
So one new row is appended for every 15 minutes in each of the .csv files (different stock symbols).
What I want is to maintain the same number of rows for each of these .csv files. In other words, as data (row) get appended to each of these .csv files, the number of rows keeps on increasing until end of the day and the following day the same routine. While I am aware of the Keep Bottom Rows, how can i do this for individual files (262 of them) through power query.
Here is the query :
let
Source = Folder.Files("C:\Yogya Data Collection\Data Between Dates"),
#"Split Column by Delimiter" = Table.SplitColumn(Source, "Name", Splitter.SplitTextByDelimiter(".", QuoteStyle.Csv), {"Name.1", "Name.2", "Name.3"}),
#"Removed Other Columns" = Table.SelectColumns(#"Split Column by Delimiter",{"Name.1", "Content"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Other Columns",{{"Name.1", "Symbol"}}),
#"Invoked Custom Function" = Table.AddColumn(#"Renamed Columns", "Transform File", each #"Transform File"([Content])),
#"Removed Columns" = Table.RemoveColumns(#"Invoked Custom Function",{"Content"}),
#"Expanded Transform File" = Table.ExpandTableColumn(#"Removed Columns", "Transform File", {"Date", "Time", "Open", "High", "Low", "Close", "Volume"}, {"Transform File.Date", "Transform File.Time", "Transform File.Open", "Transform File.High", "Transform File.Low", "Transform File.Close", "Transform File.Volume"}),
#"Renamed Columns1" = Table.RenameColumns(#"Expanded Transform File",{{"Transform File.Date", "Date"}, {"Transform File.Time", "Time"}, {"Transform File.Open", "Open"}, {"Transform File.High", "High"}, {"Transform File.Low", "Low"}, {"Transform File.Close", "Close"}, {"Transform File.Volume", "Volume"}})
in
#"Renamed Columns1"
Any help would be highly appreciated.
Much Regards,
~GK
See Answer
July 16, 2010
Offline
Hi GK,
Welcome to our forum!
If you want the action performed at the file level before appending the files together you make the transformations in the "Transform Sample File from..." query. All steps you apply in this query are applied to all files before appending.
Mynda
Answers Post
April 2, 2021
Offline
Awesome Mynda, saved a lot of my time, been mulling over this since the past few days, really appreciate it.
Now you just opened up the can of worms, I am going bug you all so much with my queries now 🙂 (on a lighter side).
While I had paid a VB guy to get my data in a specific format, it is fine, but not too happy with the way and I love Power query.
If I want to extract certain time frame out of each file (each symbol, as each symbol is saved as a separate .csv file), to be more precise, let me say I want to get values (Open-high-low-close values) for each stock (or symbol), then how can i do this in power query.
I did try doing this, but the challenge is since the data is for two days (thanks to you, else I would be looking at zillions of rows), and when I chose any specific time frame for e.g. 15:15 and 15:30 hrs then I get the data (OHLC Values) for both the days for this time unit. But I would want 15:15 and 15:30 hrs OHLC values only for yesterday and for today I will need data for let us say 9:15, 9:30 and 9:45 hours (OHLC values). How can I do this.
Let me know if you want me to attach the file and I shall do so. (as I want to learn, I am not attaching the file).
Warm Regards
GK
July 16, 2010
Offline
Hi GK,
Glad I could help.
Please post your new question as a new topic and include a sample file. Note: please don't link the sample file to a CSV file as the source data. Just put the source data in the same Excel file as your query and be sure to show us your desired result so we know exactly what you're trying to achieve.
Mynda
1 Guest(s)
