Programming (24)   Page 2
April 16, 2009

Putting a separator between your elements

Often when I’m programming, I’ll need to list a bunch of stuff and put commas between them. Like this:

1,2,3,4,5

Usually, this is what I, and most of my colleagues do:

            List nums = new List();
            for (int i = 0; i < 5; i++)
            {
                nums.Add(i);
            }

            string msg = "";
            bool first = true;
            foreach (int num in nums)
            {
                if (first)
                {
                    first = false;
                }
                else
                {
                    msg += ",";
                }
                msg += num.ToString();
            }

After reading Eric Lippert's little article about the horrid-seeming problem of commas and lists, I noticed him mentioning the method

String.Join

.

Needless to say, I went and tried it, and found that I could do the equivalent of the above, like this:

            List nums = new List();
            for (int i = 0; i < 5; i++)
            {
                nums.Add(i);
            }

            string msg = string.Join(",", nums.Select(x => x.ToString()).ToArray());
            Console.WriteLine(msg);

Its kinda amazing how after coding in C# for a good year now that I haven't noticed the existence of such a method.

Okay, the lambda expression seems a bit messy with all the parens and all, but you get the idea. Still, its a lot better than writing a loop and having the evil "first" variable which kinda clutters the whole thing. Does anyone know an even better way of doing this?

Update: and for my answer to Eric's next post:

    string msg = "{";
    IEnumerable s = something;

    string[] arr = s.ToArray();

    if (arr.Length > 1)
    {
        string[] ar2 = {
                   string.Join(", ", arr, 0, arr.Length - 1 ),
                    arr[arr.Length - 1]
               };

        msg += string.Join(" and ", ar2);
    }
    else if (arr.Length == 1) msg += arr[0];

    msg += "}";
last comments  
Ryan A
Ryan A

Use a functional language, 1 function takes list, has two cases... it's like 3 lines of code. Something like fun {Join ...
March 20, 2009

Updated the b_pool!

I’ve updated the b_pool again! This one has got some new features such as string functions (not yet complete) and a new b_resize function, which basically acts the same as realloc, except that if you are resizing the last block created on the pool, it will resize it without freeing. This makes it a lot faster and is perfect for string functions. I haven’t tested it yet and its too late at night for me to test it. If anyone would be so keen to write some tests that will break the b_resize function?

March 11, 2009

Requesting Code Review

So I’ve been working on a memory pool lately as a hobby project, and I’ve come across some things that I haven’t found a solution that would satisfy my gut yet. But, after reading this:

I started to realize that since I’m already on the internet, why not have people examine my code instead? To that effect, I’m asking everyone who’s not afraid of C and actually reads this blog to comment on my code. It can be found here:

http://bazaar.launchpad.net/~davidsiaw/bpool/trunk/annotate/head%3A/b_pool//b_pool.c

This is the center of the code development at the moment. I’ll add more comments later on how the pool is actually supposed to work. I’d like to know if there are any bad practices catchable from the outset. Thanks!

March 10, 2009

A resource pool implementation

After some interesting discussion with a friend, I’ve decided to try my hand at coding a resource pool like Apache’s APR pools. I’m gonna call my pool b_pool, which stands for bunny pool.

Its not going to be a copy of APR pools however, it will just by my idea of what a pool should be like. It will be able handle not only memory, but file handles, socket descriptors and other shared system resources and manage them. Thus, instead of calling it just a memory pool, I call it a resource pool.

You can check it out from LaunchPad using Bazaar version control.

bzr branch lp:~davidsiaw/bpool/trunk

Currently the only way to build bpool is by using Visual Studio 2008. I’ll check in some autogen scripts sometime later this week. Or someone can help me write a temporary makefile that I can merge into the tree ^_^.

This resource pool is meant to be used inside a subsystem that has a limited lifetime, but uses memory in a complex enough way to make tracking the lifetimes of objects allocated inside it difficult. The idea of a pool is that you create a pool just before you run the subsystem, and the subsystem will use the pool to allocate memory and other resources, in other words, allocate resources inside the pool. When the subsystem is no longer needed, the pool is destroyed, taking along with it all the resources it was used to allocate. The subsystem has the freedom to free memory if it wants to, or it can allocate memory to be used at a later time.

The b_pool is still in its infancy and I still have a lot of things I want to add to it such as string manipulation functions that use the b_pool, which would be tremendously useful if used together with a subsystem that uses strings a lot.

How do you use the pool?

Well right now there’s only one way to use it, and it can only pool memory at the moment. In order to create a pool, simply create a variable to hold a pointer to the pool and call b_create_pool

b_pool* pool;
b_create_pool(&pool);

When you are sure you are finished with it, you call

b_destroy_pool(pool);

This frees all the memory allocated in the pool.

In order for the subsystem to allocate memory in the pool, it needs to call the function b_alloc, which is basically a replacement for malloc, but it is used slightly differently.

void mysubsystem(b_pool* pool)
{
char* mem1;
b_alloc(pool, &mem1, 50);
}

b_alloc takes 3 parameters, it takes the pointer to the pool, a reference to the pointer to memory that it will assign to, and the size of memory requested. b_alloc will then fill in the pointer with an address where the requested memory has been prepared. It returns a number too, 0 for success and anything else for fail. I might add an enumeration later.

If you so wish to free the memory because you know you are finished with it, you can use

b_free(pool, mem);

Similar to free() in libc, b_free frees up memory to be allocated by somebody else again. Although this is optional as long as you destroy the pool when your subsystem has done its job, this is useful whenever you want to keep the memory usage to a minimum constantly, which is the case in memory-constrained environments like mobile phones.

So whats the difference between using a memory pool and malloc to allocate memory? Isn’t it just putting a big malloc and free around a set of function calls? It may seem absurd and unuseful on the outset, but the only rule you must definitely observe when using a pool is that you must Never make a global pool in your application, because that’s what your application address space already is. Essentially, yes. It is putting a big malloc and free around a set of function calls. But the usage of memory is simplified in the sense that you can ignore having to worry about trying to reuse memory that isn’t used anymore inside your subsystem, or resizing the amount of memory you allocated for your subsystem in order to let it store more data. All you need to do is ask for memory, which is one less worry in a low-level language.

It is also faster to have a userland routine manage your memory for you, since it does not incur the cost of changing privilege levels, which is expensive on most CPU architectures, and is required whenever you want to ask the OS to reserve memory for you. Eventually I will create more routines to allow the registration of destructors, deep copiers for other kinds of custom resources into the pool. I am also thinking of turning it into a layer over a set of OS resource requesting functions, so one can write subsystems that are platform-independant, and all one needs to do to use a subsystem on another platform is to implement the b_pool for that platform and use the subsystem.

So, have fun with experimenting with it and tell me what you think about it, or even, tell me your idea of what you think a resource pool should be like!

last comments  
Ryan A
Ryan A

I could see this thing being threaded with it's own form of allocation resizing and dare I say it, garbage ...
February 7, 2009

Let’s write asm!

Today I was asked by a friend to help with his programming assignment. His program needs to take in a number and output the prime factors of the number. For example, 60 = (2^2)(3^1)(5^1)

So after trying to explain the problem to him, I decided to try writing it myself in C. Of course, he wrote his own version in Java.
(more…)

January 28, 2009

What’s your favorite language?

A friend asked me what my favorite language is today, and showed me a page about parallels drawn between computer languages and religons. I told him I didn’t have a favorite language anymore. I did once upon a time, and that language was C++. But the reason it was my favorite language was because it was the only language I knew at the time. Well, not really, I knew 1 other language, PHP, but I didn’t use it much, so it was still C++.

Now, after working for about a year now, I realized that different languages are useful for different things, much like a hammer is useful for hammering nails, but not a screwdriver. The theory is that you can use a screwdriver to hammer nails too, but it wouldn’t be as effective as a hammer. The same thing applies to languages. C is good for writing OSes, and making some really fast code, but if you really don’t care and just want a script that reads a bunch of files and makes a list of filenames and e-mails it to someone, you could just use Python. Going by that analogy, you could think of each language you know as a tool in your toolbox, and the time you take to learn those languages is as well invested as the money you use to buy your hammer and sickle.

Most people have a favorite language because of the same reason I once did, because its the only language they know well. But, sometimes it may be the language of choice for what they do too. Favoritism is something I can’t really find in the languages I know now. Perhaps there would be a language that would intrigue me enough someday to make me a believer of a favorite again?

last comments  
Ryan A
Ryan A

Yea, languages are arbitrary within the context of a design paradigm. I don't particularly have a favorite either, but I ...
November 27, 2008

WordLaunch update

Some personal problems this week have delayed development but I’m not one with a faint heart! This week’s project is still going. Here is version 0.1! It allows you to edit your pages, and it probably still needs a lot of work before its any good at solving my rant that started this project, but I guess this is a start.

Here’s a video on what the program currently does:

[flashvideo filename=/wp-content/uploads/2008/11/wordlaunch-01.flv /]

(not the best video in the world, but it should illustrate what you should be clicking =P)
Theres no audio in this video, so don’t worry about letting your boss hear it =)

In order to compile this project, you will need Visual Studio 2008 and the .NET framework (duh). Also, you’ll need to unzip xulrunner into a folder and place it in the same directory as the binary.

Since not everyone has the priviledge of having a MSFT compiler, I’m also providing binaries as a download. But I won’t be responsible if they so happen to be dodgy.

Binaries

Sources

I used some libraries in this program so some credits are in order:
1. XML-RPC Library courtesy of Cook Computing
2. XULRunner courtesy of Mozilla
3. Thanks to Joseph Scott for his documentation of the XML-RPC WordPress interface.

last comments  
astrobunny
astrobunny

I think so. Express editions should work. Someone could help me try it with mono too
Ryan A
Ryan A

sweet looks nice and clean ... also, ppl can probably use the express editions no?
November 23, 2008

This week’s project: Project WordLaunch

After having some browser issues, I found that I wasn’t really happy with the way WordPress let me go about managing my pages. It was difficult to set the parent-child relationships between pages and doing massive changes. What I started to look around for was a blogging client that would allow me to perform all the changes I wanted on my computer, natively, and then upload the changes to WordPress. Unfortunately, I wasn’t able to find any such thing. So I decided to comb through WordPress’s XML RPC API, and start looking for ways to implement such a client.

last comments  
Ryan A
Ryan A

The WP XML RPC is pretty huge right ... but plenty of good stuff in there.
November 1, 2008

C is the new Assembler

As far back as I can tell, while I have used C, there was never a time when I didn’t have to write abstractions to make it easier to see that I am doing something in particular. Every knowledgeable C programmer would know that abstractions tend to make code easier to read, reducing the cost of maintenance, but it also incurs a performance penalty, as a result of data structure translations and additional function calls.

void ReadArray(Array arr, PopulateCallback cb, void* data)
{
	for (int i=0; i

This is because C was written to make assembler easier to write accurately for whatever machine it was compiled for. It was a method to allow UNIX to present the same face to the programmer no matter what machine it ran on, by abstracting commonly used concepts when dealing with hardware and operating systems. Thus the concept of pointers, packing, memory management and low-level optimizations are important in the language.

#define INITIALSIZE 32
Hashtable* CreateHashtable()
{
	Hashtable* h = malloc(sizeof(Hashtable));
	h->array = malloc(sizeof(INITIALSIZE));
	h->arraysize = INITIALSIZE;
	h->hashfun = &implePrimeHashFunc;
	return h;
}

void DeleteHashtable(Hashtable* h)
{
	free(h->array);
	free(h);
}

However, that is operating systems. Fast forward to the modern real world, which requires programs to
easily maintainable, and highly flexible when responding to code changes, layer after layer of abstraction is piled on top of each other to make the seemingly mundane tasks of sorting arrays and creating hashtables, or even responding to a user's keyboard possible. The reason for this is the low levelness of C. C does not have first class support for concepts that have arisen from real-world requirements and sensible programming techniques, such as closures, currying, lambdas and automated memory management.

class Program
{
	static void Main (string[] args)
	{
		Dictionary; mapOfEmployeeNumberToAge = new Dictionary();
		mapOfEmployeeNumberToAge.Add(5,6);
	}
}

This was because operating systems never had the luxury of being supported by a complete set of libraries. They have to stand on their own and can only depend on themselves. This meant that if one were to write an operating system for many machines, one needed an abstract form of assembler to be able to write for many platforms. Because C allowed OSes to present the same face across all platforms, it seemed the perfect choice at the time to write in C too, and it probably was.

FILE* fp = fopen("five.txt","w");
fprintf(fp, "%d", 5);
fclose(fp);

Looking at things this way, C is really just an abstract assembler, abstracting the single instructions into recognizable concepts such as pointers and associating them with types to reduce mistakes. In reality, nothing has changed in software development world ever since the 1970's when UNIX was conceived. People are still writing in assembler. There is nothing wrong with it, but the tediousness of having to abstract simple tasks has driven up the cost of writing software lately as complexity has increased tremendously.

short a = 0x1234;
short b = a;
   mov ax, 1234h
   mov bx, ax

However, mankind has probably learned the hard way that eventually there was need for better languages, and came up with many different languages such as Java, C++, C#, D, Processing, Python. All of which attempt to improve on C by introducing concepts absent in C that were highly desired by programmers. However, in an egoistic attempt to reach for the stars, they have all been made to compile directly to assembler, and thus end up still less portable than C and also perform slower as a result of those abstractions. In almost 40 years of development with C, it can be said that there is no other language more portable and no other language whose compilers are more capable of optimization than those for C.

error CS0006: cannot find metadata file `System.Windows.Forms.dll'Compilation failed:

One comes to wonder why, if there already are so many languages that can cover the shortcomings of C, they still cannot interoperate with C without an incredibly large amount of glue code and abstraction systems? All this makes for an extremely difficult situation which people have to bear with when they write for portability. Perhaps it is because the desires of proprietary lock ins done by companies such as Microsoft to promote this trend, to push their products such as COM/OLE and .NET which they preach as superior interop systems, yet fall short of achieving the very thing they are made to do, in order to prevent interop outside of Windows.

class NativeHashtable
{
	[DllImport("mydll")]
	static extern IntPtr CreateHashTable();

}

I propose that we leverage the existing codebase and toolchains that have made C so widely used and so well supported to make it simpler to write portable programs. This has many advantages: First, by targeting a subset of C that can be compiled by any existing C compiler at all, one effectively moves the repetitous task of writing abstractions and custom data structures to provide a code base from which programs can work on to the compiler specific to the language, and allowing the C compiler to provide the powerful optimization capabilites. Such a language that targets C will also find itself in a good position, since C already is compilable on many platforms, a language that targets C will be compilable too on many platforms.

	int NearestPow2(int n)
	{
		int x = 1;

		while(x < n) {
			x <<= 1;
		}

		return x;
	}

Also, compilers written for another language can perform their own optimizations in C code, which is what humans do, and maintain the rest of the program's data structures instead of having a human being do it, which is error prone and usually inconsistent. It also allows a program to write code effectively with one convention and allows one to automate the task of documenting hacks, tasks and tricks such as flag-reading, wrapping function pointers and checking datatypes, and maintaining structures and functions which use them, which is usually a choke point that requires extreme discipline from programmers and usually is tedious and takes up a lot of time.

// Populate an array.
// The PopulateCallback will be called as many times as there are elements in the array
// It will be given a pointer to a particular point in the array and it is cb's duty to modify
// the contents of each cell.
// data is any generic data that you can provide your cb for its use.
void PopulateArray(Array arr, PopulateCallback cb, void* data);

Writing such a compiler would also be easier than writing a compiler that targets assembler. There are more people who know C than people who know assembler, and it is an easy language to write automatically, since the same things are done over and over again when we write C code, in the form of maintaining declarations and definitions.

In order to target C for existing C compilers, one cannot realistically automate writing code for pure ANSI C. Instead, one must target a defined subset of C that resolves into machine code in a predictable way. There are many definitions out there such as C-- and CIL, however none of them have been put to significant practical use.

astrobunny@localhost$ ./program.cexe
C:\> cil program.cexe

If C were ever to be a target for modern languages, and used as an intermediate language for just-in-time compilation by compilers which have already abstracted system calls specific to a platform, one may finally be able to achieve the task that portability proponents have always longed for, with the performance to boot. C is the most portable language on Earth. Why not make the most of it?

last comments  
ryan aloevera gel
ryan aloevera gel

Good points, C is uber important, though a C for the future may be something like objective-C, which is C ...
October 19, 2008

RBSP Tree for Lightmap packing

For fun, and for the purpose of making a batch texture that can store characters for fonts, I have created a little tree based off this tutorial by BlackPawn. Basically, the idea here is to partition the original rectangle in such a way to get the maximum area out of the texture to hold the subtextures.

The pseudocode provided in BlackPawn’s site is based off a simple concept. Basically, you partition an empty rectangle to store an image, and the other resulting empty rectangles are candidates for future partitioning. I call this structure a Rectangular Binary Space Partitioning Tree.

(more…)

«page 2 of 3 »