Canonical Voices

Posts tagged with 'file'

mandel

In the last post I explained how to set the security attributes of a file on Windows. What naturally follows such a post is explaining how to implement the os.access method that takes into account such settings because the default implementation of python will ignore them. Lets first define when does a user have read access in our use case:

I user has read access if the user sid has read access our the sid of the ‘Everyone’ group has read access.

The above also includes any type of configuration like rw or rx. In order to be able to do this we have to understand how does Windows NT set the security of a file. On Windows NT the security of a file is set by using a bitmask of type DWORD which can be compared to a 32 bit unsigned long in ANSI C, and this is as far as the normal things go, let continue with the bizarre Windows implementation. For some reason I cannot understand the Windows developers rather than going with the more intuitive solution of using a bit per right, they instead, have decided to use a combination of bits per right. For example, to set the read flag 5 bits have to be set, for the write flag they use 6 bits and for the execute 4 bits are used. To make matters more simple the used bitmask overlap, that is if we remove the read flag we will be removing bit for the execute mask, and there is no documentation to be found about the different masks that are used…

Thankfully for use the cfengine project has had to go through this process already and by trial an error discovered the exact bits that provide the read rights. Such a magic number is:

0xFFFFFFF6

Therefore we can easily and this flag to an existing right to remove the read flag. The number also means that the only import bit that we are interested in are bits 0 and 3 which when set mean that the read flag was added. To make matters more complicated the ‘Full Access’ rights does not use such flag. In order to know if a user has the Full Access rights we have to look at bit 28 which if set does represent the ‘Full Access’ flag.

So to summarize, to know if a user has the read flag we have to look at bit 28 to test for the ‘Full Access’ flag, if the ‘Full Access’ was not granted we have to look at bits 0 and 3 and when both of them are set the usre has the read flag, easy right ;) . Now to the practical example, the bellow code does exactly what I just explained using python and the win32api and win32security modules.

from win32api import GetUserName
 
from win32security import (
    LookupAccountName,
    LookupAccountSid,
    GetFileSecurity,
    SetFileSecurity,
    ACL,
    DACL_SECURITY_INFORMATION,
    ACL_REVISION
)
from ntsecuritycon import (
    FILE_ALL_ACCESS,
    FILE_GENERIC_EXECUTE,
    FILE_GENERIC_READ,
    FILE_GENERIC_WRITE,
    FILE_LIST_DIRECTORY
)
 
platform = 'win32'
 
EVERYONE_GROUP = 'Everyone'
ADMINISTRATORS_GROUP = 'Administrators'
 
def _int_to_bin(n):
    """Convert an int to a bin string of 32 bits."""
    return "".join([str((n >> y) & 1) for y in range(32-1, -1, -1)])
 
def _has_read_mask(number):
    """Return if the read flag is present."""
    # get the bin representation of the mask
    binary = _int_to_bin(number)
    # there is actual no documentation of this in MSDN but if bt 28 is set,
    # the mask has full access, more info can be found here:
    # http://www.iu.hio.no/cfengine/docs/cfengine-NT/node47.html
    if binary[28] == '1':
        return True
    # there is no documentation in MSDN about this, but if bit 0 and 3 are true
    # we have the read flag, more info can be found here:
    # http://www.iu.hio.no/cfengine/docs/cfengine-NT/node47.html
    return binary[0] == '1' and binary[3] == '1'
 
def access(path):
    """Return if the path is at least readable."""
    # for a file to be readable it has to be readable either by the user or
    # by the everyone group
    security_descriptor = GetFileSecurity(path, DACL_SECURITY_INFORMATION)
    dacl = security_descriptor.GetSecurityDescriptorDacl()
    sids = []
    for index in range(0, dacl.GetAceCount()):
        # add the sid of the ace if it can read to test that we remove
        # the r bitmask and test if the bitmask is the same, if not, it means
        # we could read and removed it.
        ace = dacl.GetAce(index)
        if _has_read_mask(ace[1]):
            sids.append(ace[2])
    accounts = [LookupAccountSid('',x)[0] for x in sids]
    return GetUserName() in accounts or EVERYONE_GROUP in accounts

When I wrote this my brain was in a WTF state so I’m sure that the horrible _int_to_bin function can be exchanged by the bin build in function from python. If you fancy doing it I would greatly appreciate it I cannot take this any longer ;)

Read more
mandel

While working on making the Ubuntu One code more multiplatform I founded myself having to write some code that would set the attributes of a file on Windows. Ideally os.chmod would do the trick, but of course this is windows, and it is not fully supported. According to the python documentation:

Note: Although Windows supports chmod(), you can only set the file’s read-only flag with it (via the stat.S_IWRITE and stat.S_IREAD constants or a corresponding integer value). All other bits are ignored.

Grrrreat… To solve this issue I have written a small function that will allow to set the attributes of a file by using the win32api and win32security modules. This solves partially the issues since 0444 and others cannot be perfectly map to the Windows world. In my code I have made the assumption that using the groups ‘Everyone’, ‘Administrators’ and the user name would be close enough for our use cases.

Here is the code in case anyone has to go through this:

from win32api import MoveFileEx, GetUserName
 
from win32file import (
    MOVEFILE_COPY_ALLOWED,
    MOVEFILE_REPLACE_EXISTING,
    MOVEFILE_WRITE_THROUGH
)
from win32security import (
    LookupAccountName,
    GetFileSecurity,
    SetFileSecurity,
    ACL,
    DACL_SECURITY_INFORMATION,
    ACL_REVISION
)
from ntsecuritycon import (
    FILE_ALL_ACCESS,
    FILE_GENERIC_EXECUTE,
    FILE_GENERIC_READ,
    FILE_GENERIC_WRITE,
    FILE_LIST_DIRECTORY
)
 
EVERYONE_GROUP = 'Everyone'
ADMINISTRATORS_GROUP = 'Administrators'
 
def _get_group_sid(group_name):
    """Return the SID for a group with the given name."""
    return LookupAccountName('', group_name)[0]
 
 
def _set_file_attributes(path, groups):
    """Set file attributes using the wind32api."""
    security_descriptor = GetFileSecurity(path, DACL_SECURITY_INFORMATION)
    dacl = ACL()
    for group_name in groups:
        # set the attributes of the group only if not null
        if groups[group_name]:
            group_sid = _get_group_sid(group_name)
            dacl.AddAccessAllowedAce(ACL_REVISION, groups[group_name],
                group_sid)
    # the dacl has all the info of the dff groups passed in the parameters
    security_descriptor.SetSecurityDescriptorDacl(1, dacl, 0)
    SetFileSecurity(path, DACL_SECURITY_INFORMATION, security_descriptor)
 
def set_file_readonly(path):
    """Change path permissions to readonly in a file."""
    # we use the win32 api because chmod just sets the readonly flag and
    # we want to have imore control over the permissions
    groups = {}
    groups[EVERYONE_GROUP] = FILE_GENERIC_READ
    groups[ADMINISTRATORS_GROUP] = FILE_GENERIC_READ
    groups[GetUserName()] = FILE_GENERIC_READ
    # the above equals more or less to 0444
    _set_file_attributes(path, groups)

For those who might want to remove the read access from a group, you just have to not pass the group in the groups parameter which would remove the group from the security descriptor.

Read more
mandel

I have been working for about 2 moths now and after releasing our internal alpha release I have found a very interesting bug. In our port to windows we have decided to try and make your live as nice as possible in an environment as rough as Windows and to do so we allow our port to auto-update to always deliver the latests bug fixes.

Ofcourse to ensure that we are updating you system with the correct data we always perform a check sum of the msi. While our msi is updating to S3 using python, the code that downloads it is C#. Here are the different codes to calcualte the checksum:

    fd = open(filename,'r')
    md5_hash = hashlib.md5(fd.read()).hexdigest()
private static string Checksum(AlgoirhtmsEnum algorithm, string filePath)
        {
            HashAlgorithm hash;
            switch (algorithm)
            {
                case AlgoirhtmsEnum.SHA1:
                    hash = new SHA1Managed();
                    break;
                case AlgoirhtmsEnum.SHA256:
                    hash = new SHA256Managed();
                    break;
                case AlgoirhtmsEnum.SHA384:
                    hash = new SHA384Managed();
                    break;
                case AlgoirhtmsEnum.SHA512:
                    hash = new SHA512Managed();
                    break;
                default:
                    hash = new MD5CryptoServiceProvider();
                    break;
 
            }
            var checksum = "";
            using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read))
            {
                var md5 = hash.ComputeHash(stream);
                for (var i = 0; i < md5.Length; i++)
                {
                    checksum += md5[i].ToString("x2").ToLower();
                }
            }
            return checksum;
        }

Believe it or not the hash returned by each piece of code was different. WTF!!!! After a ridiculous time looking at it I managed to spot the issue. If you are using python on windows, unless you use the b option when opening a file, python will convert all the CRLF to LF making the hash to be different, how to fixed this? simply open the file this way:

fd = open(filename,'rb')

Go an figure….

Read more