Overloading Operator new and delete 2

2 January 2017

Contents[Show]

I overloaded in the last post operator new and delete. Therefore, finding memory leaks and getting the first hint of the bad guys. My solution had two not-so-nice properties. With this post, I will overcome them.

What were the not-so-nice properties of my last post? At first, I got only a hint that my memory was lost; second, I had to prepare the whole bookkeeping of memory management at compile time. Those who want to know the details of these shortcomings should read the last post. But now the ugliness will go.

Who is the bad guy?

Particular tasks ask for special forces. So I have to use a small macro for debugging purposes.

I want to stress this point. I'm not a friend of macros.

Let's have a look at the macro. #define new new(__FILE__, __LINE__)

The macro causes each new call will be mapped to the overloaded new call. This overloaded new call adds the name of the file and the line number, respectively. That's exactly the information I need.

But what will happen if I use the macro in line 6?

// overloadOperatorNewAndDelete2.cpp

#include "myNew4.hpp"
// #include "myNew5.hpp"

#define new new(__FILE__, __LINE__)

#include <iostream>
#include <new>
#include <string>

class MyClass{
  float* p= new float[100];
};

class MyClass2{
  int five= 5;
  std::string s= "hello";
};

int main(){
    
    int* myInt= new int(1998);
    double* myDouble= new double(3.14);
    double* myDoubleArray= new double[2]{1.1,1.2};
    
    MyClass* myClass= new MyClass;
    MyClass2* myClass2= new MyClass2;
    
    delete myDouble;
    delete [] myDoubleArray;
    delete myClass;
    delete myClass2;
    
    dummyFunction();
    
    getInfo();
    
}

The preprocessor substitutes all new calls. That shows precisely the modified main function.

class MyClass{
  float* p= new("overloadNewAndDelete.cpp", 14) float[100];
};

class MyClass2{
  int five= 5;
  std::string s= "hello";
};

int main(){

    int* myInt= new("overloadNewAndDelete.cpp", 24) int(1998);
    double* myDouble= new("overloadNewAndDelete.cpp", 25) double(3.14);
    double* myDoubleArray= new("overloadNewAndDelete.cpp", 26) double[2]{1.1,1.2};

    MyClass* myClass= new("overloadNewAndDelete.cpp", 28) MyClass;
    MyClass2* myClass2= new("overloadNewAndDelete.cpp", 29) MyClass2;

    delete myDouble;
    delete [] myDoubleArray;
    delete myClass;
    delete myClass2;

    dummyFunction();

    getInfo();

}

Lines 2 and 12 show that the preprocessor substitutes the constants __FILE__ and __LINE__ in the macro. But how does the magic work? The header myNew4.hpp solves the riddle.

// myNew4.hpp

#ifndef MY_NEW4
#define MY_NEW4

#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <new>
#include <array>

int const MY_SIZE= 10;

int counter= 0;

std::array<void* ,MY_SIZE> myAlloc{nullptr,};

void* newImpl(std::size_t sz,char const* file, int line){
    void* ptr= std::malloc(sz);
    std::cerr << file << ": " << line << " " <<  ptr << std::endl;
    myAlloc.at(counter++)= ptr;
    return ptr;
}

void* operator new(std::size_t sz,char const* file, int line){  
    return newImpl(sz,file,line);
}

void* operator new [](std::size_t sz,char const* file, int line){  
    return newImpl(sz,file,line);
}

void operator delete(void* ptr) noexcept{
    auto ind= std::distance(myAlloc.begin(),std::find(myAlloc.begin(),myAlloc.end(),ptr));
    myAlloc[ind]= nullptr;
    std::free(ptr);
}

#define new new(__FILE__, __LINE__)

void dummyFunction(){
    int* dummy= new int;
}

void getInfo(){
    
    std::cout << std::endl;
     
    std::cout << "Allocation: " << std::endl;
    for (auto i: myAlloc){
        if (i != nullptr ) std::cout << " " << i << std::endl;
    }
    
    std::cout << std::endl;
    
}

#endif // MY_NEW4

I implement in line 25 and 29 the special operators new and new[] that delegates their functionality to the helper function newImpl (lines 18 - 23). The function does two important jobs. At first, it displays to each new call the name of the source file and the line number (line 20); second, it keeps in the static array myAlloc track of each used memory address (line 21). This fits the behavior of the overloaded operator delete that sets all memory addresses to the null pointer nullptr (line 35). The memory addresses stand for the deallocated memory areas. In the end, the function getInfo displays the memory addresses that were not deallocated. You can directly see them together with the file name and line number.

Modernes C++ Mentoring

Be part of my mentoring programs:

- - - - "Fundamentals for C++ Professionals" (open)
- - - - "Design Patterns and Architectural Patterns with C++" (open)
        
        "C++20: Get the Details" (July 2023)

Do you want to stay informed about my mentoring programs: Subscribe via E-Mail.

Of course, I can directly apply the macro in the file myNew4.hpp. That was a lot of theory. What's the output of the program?

overloadNewAndDelete

The memory areas to the memory address 0x8c3010, 0x8c3090, and 0x8c3230 were not deallocated. The bad guys are the new calls in line 24 and line 14 (overloadNewAndDelete.hpp) and the new call in line 42 (myNew4.hpp).

Impressed? I assume, yes. But the presented technique has two drawbacks. One minor and one major.

I have to overload the simple operator new and the operator new [] for arrays. This, because of overloaded operator new, is not a fallback for the three remaining operators new.
I can not use the particular operator new that returns in the error case a null pointer. Because it will be explicitly called by the operator new with the argument std::nothrow: int* myInt= new (std::nothrow) int(1998);

Now, I have to solve the first issue. I want to use for the array myAlloc, a data structure that manages its memory at run time. Therefore, it is not necessary anymore to eagerly allocate the memory at compile time.

All at run time

What was the reason that I couldn't allocate memory in the operator new? The operator new was globally overloaded. Therefore, a call of new would end in a never-ending recursion. That will happen exactly if I use a container like std::vector that dynamically allocates its memory.

This restriction holds not anymore because I didn't overload the global operator new. That is a fallback to the three remaining new operators. Thanks to the macro, my variant of operator new is used. Therefore, I can use std::vector in my operator new.

Exactly that can you see in my program.

// myNew5.hpp

#ifndef MY_NEW5
#define MY_NEW5

#include <algorithm>
#include <cstdlib>
#include <iostream>
#include <new>
#include <string>
#include <vector>

std::vector<void*> myAlloc;

void* newImpl(std::size_t sz,char const* file, int line){
    static int counter{};
    void* ptr= std::malloc(sz);
    std::cerr << file << ": " << line << " " <<  ptr << std::endl;
    myAlloc.push_back(ptr);
    return ptr;
}

void* operator new(std::size_t sz,char const* file, int line){  
    return newImpl(sz,file,line);
}

void* operator new [](std::size_t sz,char const* file, int line){  
    return newImpl(sz,file,line);
}

void operator delete(void* ptr) noexcept{
    auto ind= std::distance(myAlloc.begin(),std::find(myAlloc.begin(),myAlloc.end(),ptr));
    myAlloc[ind]= nullptr;
    std::free(ptr);
}

#define new new(__FILE__, __LINE__)

void dummyFunction(){
    int* dummy= new int;
}

void getInfo(){
    
    std::cout << std::endl;
     
    std::cout << "Allocation: " << std::endl;
    for (auto i: myAlloc){
        if (i != nullptr ) std::cout << " " << i << std::endl;
    }
    
    std::cout << std::endl;
    
}

#endif // MY_NEW5

I use in lines 13, 19, and 33 std::vector.

What's next?

In the next post, I will have a closer look at std::allocator. In particular, I'm interested how memory requests can be mapped to particular memory areas.

Thanks a lot to my Patreon Supporters: Matt Braun, Roman Postanciuc, Tobias Zindl, G Prvulovic, Reinhold Dröge, Abernitzke, Frank Grimm, Sakib, Broeserl, António Pina, Sergey Agafyin, Андрей Бурмистров, Jake, GS, Lawton Shoemake, Animus24, Jozo Leko, John Breland, Venkat Nandam, Jose Francisco, Douglas Tinkham, Kuchlong Kuchlong, Robert Blanch, Truels Wissneth, Kris Kafka, Mario Luoni, Friedrich Huber, lennonli, Pramod Tikare Muralidhara, Peter Ware, Daniel Hufschläger, Alessandro Pezzato, Bob Perry, Satish Vangipuram, Andi Ireland, Richard Ohnemus, Michael Dunsky, Leo Goodstadt, John Wiederhirn, Yacob Cohen-Arazi, Florian Tischler, Robin Furness, Michael Young, Holger Detering, Bernd Mühlhaus, Matthieu Bolt, Stephen Kelley, Kyle Dean, Tusar Palauri, Dmitry Farberov, Juan Dent, George Liao, Daniel Ceperley, Jon T Hess, Stephen Totten, Wolfgang Fütterer, Matthias Grün, Phillip Diekmann, Ben Atakora, Ann Shatoff, and Rob North.

Thanks, in particular, to Jon Hess, Lakshman, Christian Wittenhorst, Sherhy Pyton, Dendi Suhubdy, Sudhakar Belagurusamy, Richard Sargeant, Rusty Fleming, John Nebel, Mipko, Alicja Kaminska, and Slavko Radman.

My special thanks to Embarcadero

My special thanks to PVS-Studio

My special thanks to Tipi.build

My special thanks to Take Up code

Seminars

I'm happy to give online seminars or face-to-face seminars worldwide. Please call me if you have any questions.

Bookable (Online)

German

Clean Code: Best Practices für modernes C++: 20.06.2023 - 22.06.2023 (Präsenzschulung, Termingarantie)
Design Pattern und Architekturpattern mit C++: 22.08.2023 - 24.08.2023 (Präsenzschulung)
Embedded Programmierung mit modernem C++ 12.12.2023 - 14.12.2023 (Präsenzschulung)

Standard Seminars (English/German)

Here is a compilation of my standard seminars. These seminars are only meant to give you a first orientation.

C++ - The Core Language
C++ - The Standard Library
C++ - Compact
C++11 and C++14
Concurrency with Modern C++
Design Pattern and Architectural Pattern with C++
Embedded Programming with Modern C++
Generic Programming (Templates) with C++

New

Clean Code with Modern C++
C++20

Contact Me

Phone: +49 7472 917441
Mobil:: +49 176 5506 5086
Mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
German Seminar Page: www.ModernesCpp.de
Mentoring Page: www.ModernesCpp.org

Modernes C++,

RainerGrimmDunkelBlauSmall

Tags: new/delete, Memory

Comments

0 #1 Duane 2017-07-20 00:05

Hi to all, how is the whole thing, I think every one
is getting more from this website, and your views are nice for
new viewers.

Quote

0 #2 Nico Josuttis 2017-12-30 12:22

Rainer, there is a significant problem in the implementation you provided:
Writing on std::cout while performing the request for new memory might itself request new memory on the heap, which results in really nasty behavior (core dumps at best).
This, for example, is the case with Visual C++.
So, don't do/teach that ;-)

Quote

0 #3 Rainer Grimm 2017-12-31 06:48

Quoting Nico Josuttis:

Rainer, there is a significant problem in the implementation you provided:
Writing on std::cout while performing the request for new memory might itself request new memory on the heap, which results in really nasty behavior (core dumps at best).
This, for example, is the case with Visual C++.
So, don't do/teach that ;-)

Damn. I totally overlooked the memory allocation of std::cout.
I will fix it.

Thanks a lot,
Nico

Quote

+1 #4 root 2018-03-11 18:29

Using std::vector will cause endless recursion no? I built a quick hashbucket structure based on malloc for it with the pointer as key.

Quote

0 #5 komobo 2020-06-30 07:35

Maybe I'm missing something, but if I do the #define new new(__FILE__,__LINE) magic, what happens when the class I'm new-ing provides it's own operator new/delete? The #define is going to (a) mess up that class's definition of new() and (b) cause the overloaded operator new() to be called because of the prototype.