ext4_ext_get_blocks() returns the number of blocks allocated with buffer
head unmapped for a read from prealloc space. This is needed so that
delayed allocation doesn't do block reservation for prealloc space
since the blocks are already reserved on disk. Mark the buffer head
unwritten. Some code paths try to read the block if the buffer_head is
not new and no uptodate. Marking the buffer head unwritten avoids this
reading.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext4_ext_get_blocks() returns number of blocks allocated with buffer
heads unmapped for a read from prealloc space. This is needed so that
delayed allocation doesn't do block reservation for prealloc space since
the blocks are already resevred on disk. Fix ext4_ext_get_blocks to not
return greater than max_blocks, since some of the code paths cannot
handle such a return value.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
ext4_fallocate needs to update file size in each transaction. Otherwise
if we crash the file size won't be seen. We were also not marking
the inode dirty after updating file size before. Also when we try to
retry allocation due to ENOSPC, make sure we reset the variable ret so
that we actually do a retry.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fail migrate if we allocated new blocks via mmap write.
If we write to holes in the file via mmap, we end up allocating
new blocks. This block allocation happens without taking inode->i_mutex.
Since migrate is protected by i_mutex and migrate expects that no
new blocks get allocated during migrate, fail migrate if new blocks
get allocated.
We can't take inode->i_mutex in the mmap write path because that
would result in a locking order violation between i_mutex and mmap_sem.
Also adding a separate rw_sempahore for protection is really high overhead
for a rare operation such as migrate.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Acked-by: Jan Kara <jack@suse.cz>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the preallocated area is small zero out the full extent
instead of splitting them. This should avoid the "write
every alternate block" problem that could grow the number
of extents dramatically.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch handles possible ENOSPC errors when writing to an
uninitialized extent in case the filesystem is full.
A write to a prealloc area causes the split of an unititalized extent
into initialized and uninitialized extents. If we don't have
space to add new extent information, instead of returning error,
convert the existing uninitialized extent to initialized one. We
need to zero out the blocks corresponding to the entire extent to
prevent uninitialized data reaching userspace.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch enables extent-formatted normal symlinks. Using extents
format allows a symlink to refer to a block number larger than 2^32
on large filesystems. We still don't enable extent format for fast
symlinks, which are contained in the inode itself.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Put the old extent details back if we fail to split the
uninitialized extent.
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The "resize" option won't be noticed as it comes after the NULL option,
so if you try to mount (or in this case remount) with that option it
won't be recognized.
Cc: <linux-ext4@vger.kernel.org>
Signed-off-by: Josef Bacik <jbacik@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently fdatasync is identical to fsync in ext3.
I think fdatasync should skip journal flush in data=ordered and
data=writeback mode when it overwrites to already-instantiated blocks on
HDD. When I_DIRTY_DATASYNC flag is not set, fdatasync should skip journal
writeout because this indicates only atime or/and mtime updates.
Following patch is the same approach of ext2's fsync code(ext2_sync_file).
I did a performance test using the sysbench.
#sysbench --num-threads=128 --max-requests=50000 --test=fileio --file-total-size=128G
--file-test-mode=rndwr --file-fsync-mode=fdatasync run
The result on ext3 was:
-2.6.24
Operations performed: 0 Read, 50080 Write, 59600 Other = 109680 Total
Read 0b Written 782.5Mb Total transferred 782.5Mb (12.116Mb/sec)
775.45 Requests/sec executed
Test execution summary:
total time: 64.5814s
total number of events: 50080
total time taken by event execution: 3713.9836
per-request statistics:
min: 0.0000s
avg: 0.0742s
max: 0.9375s
approx. 95 percentile: 0.2901s
Threads fairness:
events (avg/stddev): 391.2500/23.26
execution time (avg/stddev): 29.0155/1.99
-2.6.24-patched
Operations performed: 0 Read, 50009 Write, 61596 Other = 111605 Total
Read 0b Written 781.39Mb Total transferred 781.39Mb (16.419Mb/sec)
1050.83 Requests/sec executed
Test execution summary:
total time: 47.5900s
total number of events: 50009
total time taken by event execution: 2934.5768
per-request statistics:
min: 0.0000s
avg: 0.0587s
max: 0.8938s
approx. 95 percentile: 0.1993s
Threads fairness:
events (avg/stddev): 390.6953/22.64
execution time (avg/stddev): 22.9264/1.17
Filesystem I/O throughput was improved.
Signed-off-by :Hisashi Hifumi <hifumi.hisashi@oss.ntt.co.jp>
Acked-by: Jan Kara <jack@suse.cz>
Cc: <linux-ext4@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch fix a panic while running fsfuzzer.
We are improperly checking the return of ext4_orphan_get.
Signed-off-by: Josef Bacik <jbacik@redhat.com>
Signed-off-by: Mingming Cao <cmm@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There are several cases where the running transaction can get buffers
added to its BJ_Metadata list which it never dirtied, which makes its
t_nr_buffers counter end up larger than its t_outstanding_credits
counter.
This will cause issues when starting new transactions as while we are
logging buffers we decrement t_outstanding_buffers, so when
t_outstanding_buffers goes negative, we will report that we need less
space in the journal than we actually need, so transactions will be
started even though there may not be enough room for them. In the worst
case scenario (which admittedly is almost impossible to reproduce) this
will result in the journal running out of space.
The fix is to only refile buffers from the committing transaction to the
running transactions BJ_Modified list when b_modified is set on that
journal, which is the only way to be sure if the running transaction has
modified that buffer.
This patch also fixes an accounting error in journal_forget, it is
possible that we can call journal_forget on a buffer without having
modified it, only gotten write access to it, so instead of freeing a
credit, we only do so if the buffer was modified. The assert will help
catch if this problem occurs. Without these two patches I could hit
this assert within minutes of running postmark, with them this issue no
longer arises.
Cc: <linux-ext4@vger.kernel.org>
Cc: Jan Kara <jack@ucw.cz>
Signed-off-by: Josef Bacik <jbacik@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Currently at the start of a journal commit we loop through all of the buffers
on the committing transaction and clear the b_modified flag (the flag that is
set when a transaction modifies the buffer) under the j_list_lock.
The problem is that everywhere else this flag is modified only under the jbd2
lock buffer flag, so it will race with a running transaction who could
potentially set it, and have it unset by the committing transaction.
This is also a big waste, you can have several thousands of buffers that you
are clearing the modified flag on when you may not need to. This patch
removes this code and instead clears the b_modified flag upon entering
do_get_write_access/journal_get_create_access, so if that transaction does
indeed use the buffer then it will be accounted for properly, and if it does
not then we know we didn't use it.
That will be important for the next patch in this series. Tested thoroughly
by myself using postmark/iozone/bonnie++.
Cc: <linux-ext4@vger.kernel.org>
Cc: Jan Kara <jack@ucw.cz>
Signed-off-by: Josef Bacik <jbacik@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
As pointed out by Sergey Vlasov, UDF implements its own version of
the CRC ITU-T V.41. Convert it to use the one in the library.
Signed-off-by: Bob Copeland <me@bobcopeland.com>
Cc: Sergey Vlasov <vsu@altlinux.ru>
Signed-off-by: Jan Kara <jack@suse.cz>
Fix two compilation warnings (and actual bugs in message formatting)
when UDF debugging is turned on.
Signed-off-by: Sebastian Manciulea <manciuleas@yahoo.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Fix mapping of blocks using VAT when it is stored in an inode.
UDF_I(inode)->i_data already points to the beginning of VAT header so there's
no need to add udf_ext0_offset(inode).
Signed-off-by: Sebastian Manciulea <manciuleas@yahoo.com>
Signed-off-by: Jan Kara <jack@suse.cz>
This patch implements parsing of metadata partitions and reading of Metadata
File thus allowing to read UDF 2.50 media. Error resilience is implemented
through accessing the Metadata Mirror File in case the data the Metadata File
cannot be read. The patch is based on the original patch by Sebastian Manciulea
<manciuleas@yahoo.com> and Mircea Fedoreanu <mirceaf_spl@yahoo.com>.
Signed-off-by: Sebastian Manciulea <manciuleas@yahoo.com>
Signed-off-by: Mircea Fedoreanu <mirceaf_spl@yahoo.com>
Signed-off-by: Jan Kara <jack@suse.cz>
According to OSTA UDF specification, only anchor blocks and primary volume
descriptors are placed on media relative to the last session. All other block
numbers are absolute (in the partition or the whole media). This seems to be
confirmed by multisession media created by other systems.
Signed-off-by: Sebastian Manciulea <manciuleas@yahoo.com>
Signed-off-by: Jan Kara <jack@suse.cz>
As we don't properly support writing to pseudooverwrite partition (we should
add entries to VAT and relocate blocks instead of just writing them), mount
filesystems with such partition as read-only.
Signed-off-by: Jan Kara <jack@suse.cz>
We didn't handle VAT packed inside the inode - we tried to call udf_block_map()
on such file which lead to strange results at best. Add proper handling of
packed VAT as we do it with other packed files.
Signed-off-by: Jan Kara <jack@suse.cz>
UDF media with VAT could have never worked because udf_fill_inode() didn't
know about case FILE_TYPE_VAT20. Fix this.
Signed-off-by: Jan Kara <jack@suse.cz>
We incorrectly (way to strictly) checked version of VAT on loading and thus
refuse to mount correct media. There are just two format versions - below 2.0
and above 2.0 and we understand both. So update the version check accordingly.
Signed-off-by: Jan Kara <jack@suse.cz>
Add <last block>+1 and <last block>-1 to a list of blocks which can be the
real last recorded block on a UDF media. Sebastian Manciulea
<manciuleas@yahoo.com> claims this helps some drive + media combinations
he is able to test.
Signed-off-by: Jan Kara <jack@suse.cz>
UDF anchor block detection is complicated by several things - there are several
places where the anchor point can be, some of them relative to the last
recorded block which some devices report wrongly. Moreover some devices on some
media seem to have 7 spare blocks sectors for every 32 blocks (at least as far
as I understand the old code) so we have to count also with that possibility.
This patch splits anchor block detection into several functions so that it is
clearer what we actually try to do. We fix several bugs of the type "for such
and such media, we fail to check block blah" as a result of the cleanup.
Signed-off-by: Jan Kara <jack@suse.cz>
This patch move processing of UDF virtual partitions close to the place
where other partition types are processed. As a result we now also
properly fill in partition access type.
Signed-off-by: Jan Kara <jack@suse.cz>
Report error when we fail to allocate memory for a bitmap and properly
release allocated memory and inodes for all the partitions in case of
mount failure and umount.
Signed-off-by: Jan Kara <jack@suse.cz>
Cleanup processing of volume descriptor sequence so that it is more readable,
make code handle errors (e.g. media problems) better.
Signed-off-by: Jan Kara <jack@suse.cz>
According to ECMA 167 rev. 3 (see 3/8.4.2.1), Anchor Volume Descriptor
Pointer should be recorded at two or more anchor points located at sectors
256, N, N - 256, where N - is a largest logical sector number at volume
space.
So we should always try to detect N on UDF volume before trying to find
Anchor Volume Descriptor (i.e. calling to udf_find_anchor()).
That said, all this patch does is updates the s_last_block even if the
udf_vrs() returns positive value.
Originally written and tested by Yuri Per, ported on latest mainline by me.
Signed-off-by: Yuri Per <Yuri.Per@acronis.com>
Signed-off-by: Pavel Emelyanov <xemul@openvz.org>
Cc: Max Lyadvinsky <Max.Lyadvinsky@acronis.com>
Cc: Vladimir Simonov <Vladimir.Simonov@acronis.com>
Cc: Andrew Neporada <Andrew.Neporada@acronis.com>
Cc: Kirill Korotaev <dev@openvz.org>
Signed-off-by: Jan Kara <jack@suse.cz>
There are several places in UDF where we declared temporary arrays of
UDF_NAME_LEN bytes on stack. This is not nice to stack usage so this patch
changes those places to use kmalloc() instead. Also clean up bail-out paths
in those functions when we are changing them.
Signed-off-by: Jan Kara <jack@suse.cz>
We don't have to check whether a directory entry already exists in a directory
when creating a new one since we've already checked that earlier by lookup and
we are holding directory i_mutex all the time.
Signed-off-by: Jan Kara <jack@suse.cz>
reorganize few code blocks in super.c which
were needlessly indented (and hard to read):
so change from:
rettype fun()
{
init;
if (sth) {
long block of code;
}
}
to:
rettype fun()
{
init;
if (!sth)
return;
long block of code;
}
or
from:
rettype fun2()
{
init;
while (sth) {
init2();
if (sth2) {
long block of code;
}
}
}
to:
rettype fun2()
{
init;
while (sth) {
init2();
if (!sth2)
continue;
long block of code;
}
}
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
remove now unneeded kernel_timestamp type with conversion functions
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
* kernel_timestamp type was almost unused - only callers of udf_stamp_to_time
and udf_time_to_stamp used it, so let these functions handle endianness
internally and don't clutter code with conversions
* rename udf_stamp_to_time to udf_disk_stamp_to_time
and udf_time_to_stamp to udf_time_to_disk_stamp
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
block cannot be less than 0, because it's sector_t,
so remove unneeded checks
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
- translate udf_file_entry_alloc_offset macro into function
- translate udf_ext0_offset macro into function
- add comment about crypticly named fields in struct udf_inode_info
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
- move all brelse(ibh) after main if, because it's called
on every path except one where ibh is null
- move variables to the most inner blocks
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
replace all:
little_endian_variable = cpu_to_leX(leX_to_cpu(little_endian_variable) +
expression_in_cpu_byteorder);
with:
leX_add_cpu(&little_endian_variable, expression_in_cpu_byteorder);
sparse didn't generate any new warning with this patch
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com
Signed-off-by: Jan Kara <jack@suse.cz>
- remove one indentation level by little code reorganization
- convert "if (smth) BUG();" to "BUG_ON(smth);"
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
- constify internal crc table
- mark udf_crc "in" parameter as const
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
- fix error handling - always zero output variable
- don't zero explicitely fields zeroed by memset
- mark "in" paramater as const
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
udf_build_ustr was broken:
- size == 1:
dest->u_len = ptr[1 - 1], but at ptr[0] there's cmpID,
so we created string with wrong length
it should not happen, so we BUG() it
- size > 1 and size < UDF_NAME_LEN:
we set u_len correctly, but memcpy copied one needless byte
- size == UDF_NAME_LEN - 1:
memcpy overwrited u_len - with correct value, but...
- size >= UDF_NAME_LEN:
we copied UDF_NAME_LEN - 1 bytes, but dest->u_name is array
of UDF_NAME_LEN - 2 bytes, so we were overwriting u_len with
character from input string
nobody noticed because all callers set size
to acceptable values (constants within range)
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
- fix error handling - always zero output variable
- don't zero explicitely fields zeroed by memset
- mark "in" paramater as const
- remove outdated comment
Signed-off-by: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
