script and description for restoring data

pull/8097/head
ulferts 5 years ago
parent cbcc4627f5
commit c42d75eb50
No known key found for this signature in database
GPG Key ID: A205708DE1284017
  1. 4
      db/migrate/20191216135213_join_table_for_active_activities.rb
  2. 2
      docs/installation-and-operations/misc/docker-postgresql-migration/README.md
  3. 124
      docs/installation-and-operations/misc/time-entries-corrupted-by-10-4/README.md
  4. 2
      docs/installation-and-operations/operation/backing-up/README.md
  5. 2
      docs/installation-and-operations/operation/restoring/README.md
  6. 106
      lib/tasks/time_entry_activities.rake

@ -57,7 +57,7 @@ class JoinTableForActiveActivities < ActiveRecord::Migration[6.0]
FROM
time_entries te_source
INNER JOIN enumerations ON te_source.activity_id = enumerations.id AND enumerations.parent_id IS NOT NULL AND enumerations.type = 'TimeEntryActivity'
WHERE
WHERE
te_sink.id = te_source.id
SQL
)
@ -67,7 +67,7 @@ class JoinTableForActiveActivities < ActiveRecord::Migration[6.0]
ActiveRecord::Base.connection.exec_query(
<<-SQL
UPDATE
time_entries te_sink
time_entries te_sink
SET
activity_id = COALESCE(child.id, root.id)
FROM

@ -64,7 +64,7 @@ To run the migration part of the image, you will have to provide two environment
Note down or copy the current MySQL `DATABASE_URL`
```bash
# Will look something something of the kind
# Will look something of the kind
# mysql2://user:password@localhost:3306/dbname
# Pass into the container but replace mysql2 with mysql!

@ -0,0 +1,124 @@
# Fixing time entries corrupted by upgrading to 10.4.0
<div class="alert alert-info" role="alert">
**Note**: This guide only concerns installations having upgraded exactly to the OpenProject version 10.4.0. Installations having upgraded to 10.4.1 directly are not affected.
</div>
The migration scripts that ran as part of the OpenProject 10.4.0 upgrade include an unfortunate bug that leads to some installations suffering data loss.
Installations, that had time entry activities enabled/disabled per project, will have all their time entries assigned to a single time entry activity.
This guide describes how to fix the data once this has happened.
## Preconditions
* A backup file of a database state prior to 10.4.0.
* Credentials with the permission to create a database in the database server the OpenProject installation is running against.
* Console access to the OpenProject server.
## 1. Create a second database from the backup
Backup scripts are by default created via the [built in OpenProject command](../../operation/backing-up).
When not following the default, the database or the OpenProject server itself may have been backed up.
This guide only covers the proceedings for the the built in backup command.
But the reader might deduce the steps neccessary to restore accordingly for a custom backup from this guide.
As a result of this step, a second database, not the database OpenProject is currently connecting to, will contain the data of the backup.
First, connect to the OpenProject server and get the necessary details about your current database:
```bash
$ openproject config:get DATABASE_URL
#=> e.g.: postgres://<dbusername>:<dbpassword>@<dbhost>:<dbport>/<dbname>
```
Example:
```bash
$ openproject config:get DATABASE_URL
postgres://openproject:L0BuQvlagjmxdOl6785kqwsKnfCEx1dv@127.0.0.1:45432/openproject
```
Using this connection string, the following command will create the database the backup will be restored to (named `openproject_backup` in this example):
```bash
$ psql "postgres://<dbusername>:<dbpassword>@<dbhost>:<dbport>/<dbname>" -c 'CREATE DATABASE <new_dbname>'
CREATE DATABASE
```
Example:
```bash
$ psql "postgres://openproject:L0BuQvlagjmxdOl6785kqwsKnfCEx1dv@127.0.0.1:45432/openproject" -c 'CREATE DATABASE openproject_backup'
CREATE DATABASE
```
Next, that newly created database will receive the data from a backup file which typically can be found in `/var/db/openproject/backup`
```bash
$ ls -al /var/db/openproject/backup/
total 1680
drwxr-xr-x 2 openproject openproject 4096 Nov 19 21:00 .
drwxr-xr-x 6 openproject openproject 4096 Nov 19 21:00 ..
-rw-r----- 1 openproject openproject 1361994 Nov 19 21:00 attachments-20191119210038.tar.gz
-rw-r----- 1 openproject openproject 1060 Nov 19 21:00 conf-20191119210038.tar.gz
-rw-r----- 1 openproject openproject 126 Nov 19 21:00 git-repositories-20191119210038.tar.gz
-rw-r----- 1 openproject openproject 332170 Nov 19 21:00 postgresql-dump-20191119210038.pgdump
-rw-r----- 1 openproject openproject 112 Nov 19 21:00 svn-repositories-20191119210038.tar.gz
```
We will need the most recently created (but before the migration to 10.4) file following the schema `postgresql-dump-<TIMESTAMP>.pgdump`.
Using that file we can then restore the database to the newly created database (called `openproject_backup` in our example). **In the following steps, ensure that you do not restore to the currently running database**.
```bash
$ pg_restore -d "postgres://<dbusername>:<dbpassword>@<dbhost>:<dbport>/<new_dbname>" /var/db/openproject/backup/postgresql-dump-<TIMESTAMP>.pgdump`
```
Example:
```bash
$ pg_restore -d "postgres://openproject:L0BuQvlagjmxdOl6785kqwsKnfCEx1dv@127.0.0.1:45432/openproject_backup" /var/db/openproject/backup/postgresql-dump-20191119210038.pgdump`
```
That command will restore the contents of the backup file into the auxillary database.
## 2. Run the script to fix the database entries
The script that fixes the time entries can then be called:
```bash
$ BACKUP_DATABASE_URL=postgres://<dbusername>:<dbpassword>@<dbhost>:<dbport>/<new_dbname> sudo openproject run bundle exec rails openproject:reassign_time_entry_activities
```
Example
```bash
$ BACKUP_DATABASE_URL=postgres://openproject:L0BuQvlagjmxdOl6785kqwsKnfCEx1dv@127.0.0.1:45432/openproject_backup sudo openproject run bundle exec rails openproject:reassign_time_entry_activities
```
The script will then print out the number of time entries it has fixed.
```bash
Fixing 341 time entries.
Done.
```
## 3. Cleanup
The database containing the backup data can be removed once the script has finished (again, **ensure to reference the auxillary database for the drop command**):
```bash
$ psql "postgres://<dbusername>:<dbpassword>@<dbhost>:<dbport>/<dbname>" -c 'DROP DATABASE <new_dbname>'
DROP DATABASE
```
Example:
```bash
$ psql "postgres://openproject:L0BuQvlagjmxdOl6785kqwsKnfCEx1dv@127.0.0.1:45432/openproject" -c 'DROP DATABASE openproject_backup'
DROP DATABASE
```

@ -34,7 +34,7 @@ The command will create backup files in the following location on your system:
/var/db/openproject/backup
```
The content of that directory should look very similar to the following (depending on your database engine, you will see either a `mysql-dump-<date>.sql.gz` or a `postgresql-dump-<date>.pgdump` file).
The content of that directory should look very similar to the following.
```bash
root@ip-10-0-0-228:/home/admin# ls -al /var/db/openproject/backup/

@ -63,7 +63,7 @@ tar xzf svn-repositories-20191119210038.tar.gz -C /var/db/openproject/svn
Note: in this section, the `<dbusername>`, `<dbhost>` and `<dbname>` variables that appear below have to be replaced with
the values that are contained in the `DATABASE_URL` setting of your
installation. This setting can be seen by running:
installation.
First, get the necessary details about your database:

@ -0,0 +1,106 @@
#-- copyright
# OpenProject is an open source project management software.
# Copyright (C) 2012-2020 the OpenProject GmbH
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version 3.
#
# OpenProject is a fork of ChiliProject, which is a fork of Redmine. The copyright follows:
# Copyright (C) 2006-2017 Jean-Philippe Lang
# Copyright (C) 2010-2013 the ChiliProject Team
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# See docs/COPYRIGHT.rdoc for more details.
#++
require 'pg'
namespace 'openproject' do
desc 'Fixes an error in the migration to 10.4 by fetching data from a backup.'
task :reassign_time_entry_activities do
unless ENV['BACKUP_DATABASE_URL']
puts <<~MSG
The 'BACKUP_DATABASE_URL' environment variable must be defined.
That variable needs to contain the connection string to the database from which the activities are to be fetched.
MSG
next
end
select_statement = <<~SQL
SELECT te_source.id, enumerations.parent_id FROM time_entries te_source INNER JOIN enumerations ON te_source.activity_id = enumerations.id AND enumerations.parent_id IS NOT NULL AND enumerations.type = 'TimeEntryActivity'
SQL
entries = begin
connection = ActiveRecord::Base
.establish_connection(ENV['BACKUP_DATABASE_URL'])
.connection
connection.select_all(select_statement)
rescue PG::ConnectionBad, ActiveRecord::NoDatabaseError, LoadError => e
puts <<~MSG
The 'BACKUP_DATABASE_URL' environment variable is incorrect. The script cannot connect to the backup database:
#{e.message}
MSG
next
ensure
connection&.close
end
unless entries.any?
puts <<~MSG
As there are no project specific activities in the backup, nothing needs to be done.
MSG
next
end
entries_string = entries.map { |entry| "(#{entry['id']}, #{entry['parent_id']})" }.join(', ')
update_statement = <<~SQL
UPDATE time_entries
SET activity_id = val.activity_id
FROM (values
#{entries_string}
) as val (id, activity_id)
WHERE time_entries.id = val.id
SQL
puts <<~MSG
Fixing #{entries.length} time entries.
MSG
connection = ActiveRecord::Base
.establish_connection(Rails.configuration.database_configuration[Rails.env])
.connection
connection.execute(update_statement)
connection.close
puts <<~MSG
Done.
MSG
end
end
Loading…
Cancel
Save