Talk:Profiling (computer programming)

Not synonym

The page used to give "program analysis" as a synonym, but that is misleading to my ears. I have clarified it to "dynamic program analysis". --Lexspoon 14:59, 9 April 2006 (UTC)

Wording change

Changed "In computer science, ..." to "In software engineering, ...". I think realistically, performance analysis is something engineers do--and only when they have to. --Jorend 12:54, 14 April 2006 (UTC)

Disam?

Shouldn't a profiling disambig be made? 134.193.168.234 23:18, 15 June 2006 (UTC) site:en.wikipedia.org profiling

Extern links just advertising

Needs to be trimmed. All the tools listed could be made into a list in the article (without the extern links), but as it is, the links offer no further info on the topic, just ads for various tools. — Frecklefoot | Talk 14:58, 15 August 2006 (UTC)

Better than none. :-) --Leo 19:44, 27 October 2006 (UTC)

Please don't delete that collection of links, maybe create a list page sorted by language and profiler features though, like list of unit testing frameworks. --Chris Pickett 01:48, 4 December 2006 (UTC)

Systems?

What about the performance analysis that Systems Administrators(/Programmers/Whatever) do?

this should be discussed in a way, and an article has to be written on Performance tuning. System Performance tuning is usually done by System Administrators it can be Server tuning, Operating system tuning to Webserver tuning, application server, database tuning depending on the component architecture of the system. Code tuning is what we call if the application or program is not tuned for performance properly, all code tuning and system tuning should be followed by load testing and retuning until the proper performance is met.— Preceding unsigned comment added by Adubinsky (talk • contribs) 11:51, 13 October 2011 (UTC)

How about static analysis?

Shouldn't this page also say something about static analysis? Maybe also a link to Algorithmic efficiency could be added. --Bernard François 21:08, 2 January 2007 (UTC)

Whither profiling?

Forgive me, but I've never seen any profiler that is nearly as effective as the simple technique of getting the program in question running under a debugger, hitting the "pause" button, and then examining the call stack. If this is done several times, if there is a slowness bug in the program, it becomes very obvious. I hate to sound like I'm voicing just a personal opinion, but this is easily put to an objective test. Any comments? 65.96.64.14 21:41, 12 March 2007 (UTC)

Low-impact tracing profilers allow you to cover far more ground quickly, and on applications that cause debuggers to choke. Example for the former is a GUI application which is sluggish, but not slow enough to allow you to switch applications to the debugger and pause. I ran into this when tuning the performance of a code editor product - it was just slow enough to be annoying, without being so slow you could apply simple tricks to it.

For the latter, consider a D3D game. Trying to pause a fullscreen game in a debugger is a nightmare, as you lose your video resources when you switch apps. If you tried to sample it with the debugger technique above, you'd be spending all your time in the driver, loading resources.

--Jim —Preceding unsigned comment added by 203.173.169.118 (talk) 03:17, 3 September 2007 (UTC)

Pardon me, but you don't need to "cover ground". Performance problems are not needles in haystacks. The worse they are, the more obvious they are. But you are right, dropping in on a program at random is not always easy to do. For the first problem, if something takes less than a second, you can put a loop around it - do it 100 times, find/fix the problem, then take away the loop. For the second problem, I've done extreme things to get call stack samples. On one project, I used an in-circuit emulator. On another, I made an "alarm-clock" signal dump the program. On another the CPU had a "halt" button on the front panel - then I could toggle through memory and read the lights. It wouldn't be worth the trouble if there were a better way. In head-to-head comparisons, "stackshots" show the problem immediately. Profilers only give statistics, graphs, and colorful displays to puzzle over. MikeDunlavey 20:30, 18 October 2007 (UTC)

I think all we've really managed to say here is that the current breed of profilers aren't doing their job properly. I'm personally quite fond of a little .NET tool that takes you straight to your performance problem. Run it up, do the slow thing, let it crunch the numbers, and `hey, here's your perf issue` a couple seconds later. Sure, you could do it by hand, but I'd rather automate the bits I can. However, I'm a profiler vendor, so I'm biased ;) --Jim —Preceding unsigned comment added by 203.97.223.50 (talk) 21:28, 23 October 2007 (UTC)

Understood, but I bet the programs you profile are either small or have lots of small functions. I'm accustomed to ugly million-liners, and profilers I've seen tell you piles of nothing. I built a tool once to automate the stack-sampling approach. It had a graph-browser style UI. It worked pretty well, but still took longer than the manual method which lets you see all the context and really understand what it's doing. Just last week I pinpointed three major problems in less time than my teammate could even think about setting up the profiler. I could say this is what it's spending X% of its time doing and maybe it doesn't need to. Result - massive speedup.

I am just trying to awaken the gut understanding that if the software is doing something unnecessary and making you wait, then it's doing it now, while you're waiting, and if you interrupt it chances are you'll catch it in the act, like an employee trying to "look busy". You don't need fine-tooth-comb detective work, intuition, cleverness, precise timing, execution counts, statistics, or graphs. What has me completely baffled is that this has not been totally obvious since day 1 of computers.

In my opinion, what the next breed of profilers should do is automate the manual technique that works, namely:

• sampling the entire call stack, not just the program counter or a limited number of levels.

• sampling only during during the interval of interest.

• taking only a moderate number of samples. 20 is enough. 100 is more than enough.

• computing for each address that appears on call stacks the percent of call stack samples it appears on. Note that this eliminates concerns about recursion. If a statement appears on a call stack sample, that counts as 1, even if it appears 10 times on that one sample.

• displays the statements in decreasing order by that percentage. Then the user looks down that list for something he/she can optimize. If a statement can be eliminated or not called, that percentage is roughly how much time will be saved.

• presentation: some sort of statement-call-graph would be OK, but not if it is only at the function level. It needs to retain call-site information. It should also let you examine individual call stack samples and the source code at each level so you can see why that particular nanosecond was being spent.

All of this is public-domain information. Anyone who wants to can build and sell such a profiler.

People will ask "Isn't this what current profilers do?" The answer is NO, they don't. They lose information right and left. For sampling, the PC alone is such a small part of the program's state that it tells you almost nothing. For instrumentation (function timing and call graph capture) it loses crucial call-site information, not to mention calls to functions that are not instrumented, like library functions. Then, to further fog the issue, they tell you lots of other stuff that you might care about but have only indirect effect on performance, such as cache misses, memory usage, and code coverage. MikeDunlavey 13:39, 11 November 2007 (UTC)

Monte Carlo Profiler?

To my knowledge "Monte Carlo Profilers" work by sampling the program counter on a timer interrupt.

Yes, and some of them also sample the entire call stack. I believe the Sun profiler is the only one (currently) that allows you to figure out, for individual call statements (as opposed to entire functions), what fraction of the time they are active on the stack.. In my experience, this is the number that matters. What's more, most of the information comes in the first few samples, the rest only giving needless precision. For example, if a program takes 20 seconds to run, and in 5 out of 10 samples the instruction "file foo.c, line 101: call bar()" appears, removing that call, if possible, would save 10 seconds (more or less). If bar() takes 10 seconds and is called once, or takes 10 microseconds and is called a million times makes no difference. Of course, removing the entire function bar() (or speeding it way up) would get the same effect, but the call is more likely to be removeable than the entire function.

To put it another way, different instructions take different amounts of time. "fadd" (floating point add) takes a lot more cycles than "mov eax,ebx" (move register). This shows up in the program counter histogram, because the timer interrupt is more likely to occur in the longer instruction. It makes sense, then, to try to use fewer of such instructions, so the program will take fewer cycles. However, some instructions take millions of times longer, and would help enormously if they could be removed, but they are essentially ignored by the program counter histogram. An example is "call _fprintf". If when a timer interrupt occurred, not only the PC were histogrammed, but every unique return address on the call stack, this deficiency would be remedied. If that slows down the program being profiled, so what? The object is to find out what needs to be optimized, not to measure how fast it isn't. MikeDunlavey 20:41, 24 May 2007 (UTC)

Time to re-org this page?

As it is, this page sort of assumes these things are the same:

• performance analysis = profiling = measuring performance = diagnosing performance problems = optimizing performance

I wonder if it makes sense to re-arrange it to split these ideas apart.

Any comments? Thanks, MikeDunlavey 14:29, 4 June 2007 (UTC)

Linkfarm

I think all the external links in the list of profilers should be removed per WP:SPAM, WP:EL, and WP:NOT#LINK. --Ronz 23:38, 25 July 2007 (UTC)

But since the tools are external, creating a page for each one would be creating an article about a product, which isn't allowed WP:SPAM. Surely an external link is more appropriate than a dead link or no link at all. Ian Broster 10:25, 23 August 2007 (UTC)

I think that there should be some kind of ordering on the links, or at least some sense of what is 'acceptable' when adding a new link:

- Alphabetical order would work.
- If someone wanted to actually evaluate all these tools, and rank them by merit, that would work too, but likely take a significant amount of effort.
- First-in, first-served also works.
- Any other well-defined ordering.

What I'm taking issue with here, is when some user (often a tool vendor) comes along and places a link to their product at the very top of the list. Frankly, its rude, and very much WP:SPAM. Instead, everyone should be considerate, and place their link either:

- In the correct place, if the list has an ordering.
- Discreetly in the middle of the list, or better, toward the bottom. Wiki isn't here to advertize your product.

-- Jim (203.173.169.118)

You can't rank the tools by merit...we would never agree on criteria :-)

Can we just go back to Alphabetical ordered links? Or follow the suggestion above from Chris Pickett to be like list of unit testing frameworks. Ian Broster 08:11, 3 September 2007 (UTC)

See WP:LIST. We can try to come up with an alternative criteria for inclusion in the list other than what I'm proposing, which is to only include tools that have their own article (or have appropriate mention in another article such as a section about a tool in an article about the company that makes the tool). --Ronz 15:29, 3 September 2007 (UTC)

Most tools cannot have their own article. Most companies cannot have their own article. We would end up with a very short list of the few tools that are famous enough (or free enough that it's not seen as advertising!) ... this would not make an informative or neutral list. The criteria for entry to the "List of profilers" is surely as simple as "Is a profiler". Nothing to do with whether the tool is GPL, or how large the company is. Ian Broster 16:19, 3 September 2007 (UTC)

Yep. Either that or we come up with another criteria. Can we find a list of these tools to refer too, preferably as part of a WP:RS? --Ronz 17:17, 3 September 2007 (UTC)

There's a couple links in the C/C++ section that don't make sense. In particular, Shark and Ants are probably not linking to the correct pages. --Jim —Preceding unsigned comment added by 203.173.169.118 (talk) 04:42, 13 September 2007 (UTC)

VTune redirects here, but is not mentioned in the text.

VTune should either not redirect here, or it should be mentioned in the text. Else, a reader still does not know in what relation VTune stands to profilers and therefore does not know what it is, which is the point of this page. —Preceding unsigned comment added by 88.77.146.24 (talk) 19:56, 6 September 2008 (UTC)

I agree. gprof (or Gprof?) has the same problem. It is mentioned though, but the redirect means there is no particular information about what Gprof is. --Mortense (talk) 15:55, 23 March 2010 (UTC)

Simple manual technique

This section does not seem to be very encyclopaedic. It describes one particular technique of performance analysis, and description is very detailed, in a how to style. I suggest removing the section completely or making it a lot briefer. Unless there will be some links provided to prove the technique is notable and not an original research, I will delete the section. The only articles linked are written by the same person which wrote this Wikipedia section, which consitutes conflict of interests. BIS Ondrej (talk) 21:42, 12 November 2008 (UTC)

Was replaced by a link to another article by the original editor, where the technique is explained briefly - which I think is great solution. Thanks. BIS Ondrej (talk) 09:36, 14 November 2008 (UTC)

Right. It's a case of gravitating away from point/counterpoint, toward "common wisdom". I guess that's the goal. MikeDunlavey (talk) 21:50, 18 May 2009 (UTC)

Microsoft link no longer works

The external link to the Microsoft article is over 4 years old, and the movie that was hosted there is no longer viewable. I commented on this on Ronz's 'talk' page, but I'm not sure if this feedback should be left here too. —Preceding unsigned comment added by 68.35.53.253 (talk) 20:59, 9 January 2009 (UTC)

Instruction level bottlenecks

Looks like another unsourced conjecture Tedickey (talk) 19:37, 27 February 2009 (UTC)

If by any chance you're referring to my papers in the references, they are neither unsourced nor conjecture.MikeDunlavey (talk) 19:31, 18 May 2009 (UTC)

Event-based profiling

I am doing some research on dynamic system analysis and wanted to list the different techniques used for profiling. Now I read here about Event-based and statistical profilers. But I cannot find on what this information is based. The only site where I could find the keyword "Event-based profiling" was this one: [1]. It's about some profiling tool. The point is that there they explain that event-based profiling is statistical profiling (Like time-based profiling, event-based profiling relies upon statistical sampling to build a program profile.), which leaves me to wonder what the basis is of the division between the two on this wikipedia page. Is this based on the intuition of the author and how he perceived it or on some other source? (This is my first comment on a wikipedia page so forgive me if I didn't keep myself to all the standards) Matthijs.Wessels (talk) 15:11, 12 October 2009 (UTC)

Random unsourced example related to DBMS_PROFILE

The example isn't encyclopedic:

• it's not sourced to a published analysis,
• there's no reliable source at all
• the bulk of it is just cut/paste (at best) with no discussion of what it does

hope that helps Tedickey (talk) 09:53, 27 February 2010 (UTC)

That's fair. Seems to me that something is better than nothing, but that's fine. An example by *somebody* would go a long way. This is truly a case of where a "picture", of sorts, is worth a thousand words. Trying to describe profiling without showing an example is like trying to describe a beautiful sunset to a blind man. Thanks for the rationale. Somewherepurple (talk) 03:19, 1 March 2010 (UTC)

Gprof should not redirect here

I don't think Gprof / gprof should redirect here. If it does not deserve its own page, then it should redirect to a list of profilers.

Convenience link: Gprof.

--Mortense (talk) 16:03, 23 March 2010 (UTC)

Agree (there are too many redirects). gprof is probably notable, but "someone" should write a topic on it to demonstrate that. Tedickey (talk) 20:56, 23 March 2010 (UTC)

What is the difference between "Instrumenting" and "Instrumentation" ?

Hello, I would like to know the difference between "Instrumenting" and "Instrumentation". There are 2 paragraphs with titles "Instrumenting profilers" and "Instrumentation". I am not sure whether "Instrumenting" and "Instrumentation" have different meaning.

Thank in advance --Wolfch (talk) 15:20, 15 December 2013 (UTC)

Hi Wolfch. I think you are correct to raise this question. I have merged the two sections. Murray Langton (talk) 17:44, 15 December 2013 (UTC)

thanks--Wolfch (talk) 02:28, 17 December 2013 (UTC)

Design and use of a program execution analyzer

One of the earlier references to a program execution analyzer is the write-up by Leigh Power, entitled "Design and use of a program execution analyzer", which appeared in the IBM SYSTEMS JOURNAL, VOL 22, NO 3, 1983, ppg 271-294. 32.97.110.58 (talk) 21:06, 12 June 2015 (UTC)Dave

A somewhat earlier reference is "An empirical study of FORTRAN programs" by D E Knuth, which appeared in Software Practice and Experience, Vol 1, pp 105-133 (1971). Murray Langton (talk) 07:16, 13 June 2015 (UTC)

There are so many articles like that, and they all make an implicit assumption: that acquiring various measurements of performance is necessary and sufficient for identifying ways to speed up code. For example in the original gprof article, it says "The profile can be used to compare and assess the costs of various implementations." It does not claim the profile can be used to identify the various implementations to be compared and assessed, even though that is generally what profilers are used for. On Stackoverflow, I've spent years telling people how to find speedups, and explaining why measurement does not accomplish it.

Here's an explanation of how easy it is for speedups to hide from most profilers: http://stackoverflow.com/a/25870103/23771. On the positive side, the technique that actually does work is given here: http://stackoverflow.com/a/378024/23771. Please note the number of votes.

It has a very simple statistical justification, that anyone can understand: http://scicomp.stackexchange.com/a/2719/1262. MikeDunlavey (talk) 01:28, 27 July 2015 (UTC)

Origin

I wonder why do we use the word "profiling". I think the origin of the term should be included in the article. — Preceding unsigned comment added by 2A02:2149:8122:C100:E815:19FD:ECD0:5E34 (talk) 08:51, 30 March 2017 (UTC)