• Skip to main content
  • Skip to header right navigation
  • Skip to site footer

My Online Training Hub

Learn Dashboards, Excel, Power BI, Power Query, Power Pivot

  • Courses
  • Pricing
    • Free Courses
    • Power BI Course
    • Excel Power Query Course
    • Power Pivot and DAX Course
    • Excel Dashboard Course
    • Excel PivotTable Course – Quick Start
    • Advanced Excel Formulas Course
    • Excel Expert Advanced Excel Training
    • Excel Tables Course
    • Excel, Word, Outlook
    • Financial Modelling Course
    • Excel PivotTable Course
    • Excel for Customer Service Professionals
    • Excel for Operations Management Course
    • Excel for Decision Making Under Uncertainty Course
    • Excel for Finance Course
    • Excel Analysis ToolPak Course
    • Multi-User Pricing
  • Resources
    • Free Downloads
    • Excel Functions Explained
    • Excel Formulas
    • Excel Add-ins
    • IF Function
      • Excel IF Statement Explained
      • Excel IF AND OR Functions
      • IF Formula Builder
    • Time & Dates in Excel
      • Excel Date & Time
      • Calculating Time in Excel
      • Excel Time Calculation Tricks
      • Excel Date and Time Formatting
    • Excel Keyboard Shortcuts
    • Excel Custom Number Format Guide
    • Pivot Tables Guide
    • VLOOKUP Guide
    • ALT Codes
    • Excel VBA & Macros
    • Excel User Forms
    • VBA String Functions
  • Members
    • Login
    • Password Reset
  • Blog
  • Excel Webinars
  • Excel Forum
    • Register as Forum Member

Time for an upgrade?|General Excel Questions & Answers|Excel Forum|My Online Training Hub

You are here: Home / Time for an upgrade?|General Excel Questions & Answers|Excel Forum|My Online Training Hub
Avatar
sp_LogInOut Log In sp_Registration Register
sp_Search Search
Advanced Search|Last Search Results
Search
Forum Scope




Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
sp_Search Search
sp_RankInfo
Lost password?
sp_CrumbsHome HomeExcel ForumGeneral Excel Questions & Answe…Time for an upgrade?
sp_PrintTopic sp_TopicIcon
Time for an upgrade?
Avatar
MATTHEW STANSBERRY
Member
Members
Level 0
Forum Posts: 7
Member Since:
October 10, 2021
sp_UserOfflineSmall Offline
1
November 18, 2021 - 9:12 pm
sp_Permalink sp_Print

Hi all! As my excel project continues to grow, my power query tables are getting monstrous.

What I typically do is append tables, add some columns, do some math, then export it into a new sheet as hard values to increase speed, rinse, and repeat. This year on my projects, I've been focusing on more vba and such which has allowed me to gather my data quickly via web query and power  query but has also made me greedy for more data which is getting cumbersome for my pc to handle. I'm not sure if it's my pc or excel in general.

What I'm asking is at what point is it time to move on and store my data now into a database and use excel to interact with that data base (assuming that is possible?) I use compact sql for my work stuff, but it's set up in such away that I have to do little under the hood manipulation with it as that is done via my work software. I've read about access a bit and have it as part of 365 but am not sure if that will help speed up performance or not. Mynda have any helpful videos on using access with excel? I'm going to have a look here in a bit. 

 

Right now my current work book is 45000KB. I'm working with 2 imported appended power query tables

one is 11 wide by 465000 tall and the other is 166 wide by 251000 tall. On my simulation Workbook when I do a full sim it does about 61 billion random numbers and some sum cells in 20 seconds, but now were talking almost 117billion cells of some math lovin. The big thing that is bogging it down is an index match (which I understand to the best of my knowledge) which is supposed to be faster to calc. than vlookup? Besides, I need the additional criteria to form my "bridge" between the two tables.

I guess what has me a bit puzzled is why the size of a work sheet is super duper big if excel isn't meant to handle it?

 

Any advice / feed back is appreciated.

 

Thanks

 

MatthewMath-is-hard.PNGImage Enlarger

sp_PlupAttachments Attachments
  • sp_PlupImage Math-is-hard.PNG (229 KB)
Avatar
Alan Sidman
Steamboat Springs, CO
Member
Members


Trusted Members
Level 0
Forum Posts: 132
Member Since:
October 18, 2018
sp_UserOfflineSmall Offline
2
November 19, 2021 - 12:39 am
sp_Permalink sp_Print sp_EditHistory

In my mind, if you have a data base then you should be using Data Base Software (ie.  MS Access) as spreadsheets are really not the most effective way to maintain it.  Spreadsheets are great for calculating and analysis but are cumbersome for data base.  My 2 cents.

BTW:  Here is a link to start if you have never used Access-->http://www.accessmvp.com/strive4peace/

Avatar
MATTHEW STANSBERRY
Member
Members
Level 0
Forum Posts: 7
Member Since:
October 10, 2021
sp_UserOfflineSmall Offline
3
November 20, 2021 - 6:33 pm
sp_Permalink sp_Print

Right on Alan thanks so much. 

 

For practice I started creating a database for my nfl football model

was able to create a schedule table

team tables with composite keys 

and one years worth of rosters and managed to create relationships between the 3.

I like how smooth it flows once you get comfortable with the navigation along with how snappy the drop down lists fire off .

Going to play around with this some more and see what all I can do within it and how I can use it more with excel and vice versa before I integrate other projects and work related material

thanks!

https://screencast-o-matic.com.....rXYj3VIuDe

 

Matthew

Avatar
Alan Sidman
Steamboat Springs, CO
Member
Members


Trusted Members
Level 0
Forum Posts: 132
Member Since:
October 18, 2018
sp_UserOfflineSmall Offline
4
November 21, 2021 - 11:28 am
sp_Permalink sp_Print sp_EditHistory

Here is a white paper that will help you as you go forward.

sp_Feed
Go to top
Forum Timezone: Australia/Brisbane
Most Users Ever Online: 245
Currently Online: Philip Treacy, Jack Brett, Natasha Smith
Guest(s) 10
Currently Browsing this Page:
1 Guest(s)
Top Posters:
SunnyKow: 1432
Anders Sehlstedt: 870
Purfleet: 412
Frans Visser: 346
David_Ng: 306
lea cohen: 219
A.Maurizio: 202
Jessica Stewart: 202
Aye Mu: 201
jaryszek: 183
Newest Members:
LAFONSO HERNANDEZ
Hayden Hao
Angela chen
Sean Moore
John Chisholm
vexokeb sdfg
John Jack
Malcolm Toy
Ray-Yu Yang
George Shihadeh
Forum Stats:
Groups: 3
Forums: 24
Topics: 6211
Posts: 27239

 

Member Stats:
Guest Posters: 49
Members: 31891
Moderators: 3
Admins: 4
Administrators: Mynda Treacy, Philip Treacy, Catalin Bombea, FT
Moderators: MOTH Support, Velouria, Riny van Eekelen
© Simple:Press —sp_Information

Sidebar

Blog Categories

  • Excel
  • Excel Charts
  • Excel Dashboard
  • Excel Formulas
  • Excel PivotTables
  • Excel Shortcuts
  • Excel VBA
  • General Tips
  • Online Training
  • Outlook
  • Power Apps
  • Power Automate
  • Power BI
  • Power Pivot
  • Power Query
microsoft mvp logo
trustpilot excellent rating
Secured by Sucuri Badge
MyOnlineTrainingHub on YouTube Mynda Treacy on Linked In Mynda Treacy on Instagram Mynda Treacy on Twitter Mynda Treacy on Pinterest MyOnlineTrainingHub on Facebook
 

Company

  • About My Online Training Hub
  • Disclosure Statement
  • Frequently Asked Questions
  • Guarantee
  • Privacy Policy
  • Terms & Conditions
  • Testimonials
  • Become an Affiliate

Support

  • Contact
  • Forum
  • Helpdesk - For Technical Issues

Copyright © 2023 · My Online Training Hub · All Rights Reserved. Microsoft and the Microsoft Office logo are trademarks or registered trademarks of Microsoft Corporation in the United States and/or other countries. Product names, logos, brands, and other trademarks featured or referred to within this website are the property of their respective trademark holders.