• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Multiple csv-files and changing content|Power Query|Excel Forum|My Online Training Hub

You are here: Home / Multiple csv-files and changing content|Power Query|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumPower QueryMultiple csv-files and changing con…
sp_PrintTopic sp_TopicIcon
Multiple csv-files and changing content
Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
1
October 8, 2020 - 5:51 pm
sp_Permalink sp_Print

Hi,

I like your examples with PowerQuery combining multiple csv-files and creating great pivots and graphs.

We started to rebuild our PerformancePortal with these techniques.

What is the best way to implement a change in file content? For instance: at some moment we want/need to have extra columns to show specific information. Is it possible to include newer columns into the 'merged' data model without having to change all previous csv-files?

 

kind regards, Maarten

sp_AnswersTopicSeeAnswer See Answer
Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
2
October 9, 2020 - 1:12 am
sp_Permalink sp_Print

Hi Maarten,

See this post for a solution, download that file attached: https://www.myonlinetraininghu.....der#p16593

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
3
October 9, 2020 - 11:39 pm
sp_Permalink sp_Print

Thanks for the response.

But: apparently it will not or cannot work for files from a SharePoint map. Do you have a likewise solution for that environment?

And I suspect that this solution deals with the adding of new files, but my real question was about file content.

For instance we have a lot of monthly files with project data, and starting from this month we want to register the responsabel account manager,

that will be registered in a new column.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
4
October 9, 2020 - 11:46 pm
sp_Permalink sp_Print

What is a SharePoint Map? Is that a sharepoint site documents?

If yes, you can simply Sync that folder locally, and you will always have files syncronized with sharepoint site. Use the local synced folder as source for extracting data.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
5
October 12, 2020 - 4:59 pm
sp_Permalink sp_Print

The power of using files from a SharePoint folder is that everyonbe with SharePoint rights can update the file; that's not possible when we use a synced folder.

But still no answer to my question: how to deal with new (necessary) columns.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
6
October 12, 2020 - 5:17 pm
sp_Permalink sp_Print

Actually, the link I provided offers a solution where it does not matter how many columns are in the source files, so you do have an answer. My question in fact does not have an answer 🙂 ("What is a SharePoint Map?")

All you have to do is to change the query to get data from sharepoint folder instead of local folder. The function that reads the files will consider any new columns added.

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
7
October 12, 2020 - 5:21 pm
sp_Permalink sp_Print

The power of using files from a SharePoint folder is that everyonbe with SharePoint rights can update the file; that's not possible when we use a synced folder.

It's about how we read the data, not how we EDIT files.
Syncing the sharepoint folder with your desktop will only change how you READ the files: you read them from the local folder, instead of reading from web.
OneDrive Sync agent will take care of bringing the online changes to your computer, so I did not suggested editing the source files OFFLINE.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
8
October 12, 2020 - 9:15 pm
sp_Permalink sp_Print

I'm sorry for the misunderstandings

  1. in Dutch a 'map' stands for a 'folder', so I mean the csv-files are in a SharePoint folder
  2. with "everyone with SharePoint rights can update the file" I mean 'update the data' within the Excel file with the Power Pivot and graphs (also located in SharePoint) , not the underlying cvs-files
  3. "The function that reads the files will consider any new columns added"; I just tried that, but the extra columns in the newest csv-files didn't show up, so what function do you mean?
Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
9
October 13, 2020 - 12:27 am
sp_Permalink sp_Print

Please upload a few sample csv files with different headers and just 1-2 rows of fake data.
Will prepare a sample.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
10
October 14, 2020 - 10:23 pm
sp_Permalink sp_Print

I have some data csv files and a simple Excel sheet to read them.

The first 3 (folder Data) are the 'standard' ones, in the folder NewData I added one with extra columns.

In the Excel sheet I got the data of the first 3 by 'reading csv from folder'.

After making pivot table I placed the 4th file and updates all data, but no extra columns.

Avatar
Philip Treacy
Admin
Level 10
Forum Posts: 1514
Member Since:
October 5, 2010
sp_UserOfflineSmall Offline
11
October 14, 2020 - 10:31 pm
sp_Permalink sp_Print

Hi Maarten,

Did you mean to attach files? You need to click the Start Upload button after selecting the files.

Phil

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
12
October 15, 2020 - 12:42 am
sp_Permalink sp_Print

Sorry, I didn't notice that!

Avatar
Catalin Bombea
Iasi, Romania
Admin
Level 10
Forum Posts: 1807
Member Since:
November 8, 2013
sp_UserOfflineSmall Offline
13
October 15, 2020 - 1:29 am
sp_Permalink sp_Print

Using the automatic file combiner is not dynamic. It ONLY reads the first file in that folder to get the headers and ignore all other headers from ALL other files.

Just use the file I already provided.

File attached here in case you don't want to download it from the other post i indicated.

Note that I had to change the csv delimiter, in your csv's the delimiter is semicolon instead of comma.

Avatar
Maarten de Keijzer
Member
Members

Power Pivot
Level 0
Forum Posts: 35
Member Since:
September 2, 2020
sp_UserOfflineSmall Offline
14
October 15, 2020 - 8:56 pm
sp_Permalink sp_Print

OK, thanks!
I will use it in our sheets.

And yes: in Dutch regional settings we always have to change things (. -> ,  and , -> 😉

sp_AnswersTopicAnswer
Answers Post
sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Shanna Henseler, Lawrence Smith
Guest(s) 10
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
John Chisholm
vexokeb sdfg
John Jack
Malcolm Toy
Ray-Yu Yang
George Shihadeh
Naomi Rumble
Uwe von Gostomski
Jonathan Jones
drsven
Forum Stats:
Groups: 3
Forums: 24
Topics: 6212
Posts: 27236

 

Member Stats:
Guest Posters: 49
Members: 31889
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.